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FOREWORD 


The  U.S.  Army  Research  Institute  Aviation  Research  and 
Development  Activity  (ARIARDA)  at  Fort  Rucker,  Alabama,  is 
contributing  to  the  effectiveness  of  Army  aviation  by  conducting 
a  comprehensive  human  factors  research  program  in  support  of 
aircrew  performance  and  training.  The  ARIARDA  research  program 
encompasses  the  full  range  of  Army  aviation,  with  projects  in 
support  of  (a)  emerging  Army  aviation  weapon  systems,  (b)  aviator 
selection,  (c)  manpower  and  personnel  integration  (MANPRINT) ,  and 
(d)  aviator  training  programs.  The  MANPRINT  and  emerging  avia¬ 
tion  systems  work  was  performed  as  part  of  the  Systems  Research 
Laboratory's  Human  Factors  Training  Operational  Effectiveness 
Project  A793 .  The  aviator  training  work  was  executed  as  part  of 
the  Training  Simulation  Project  A795  in  the  Training  Research 
Laboratory.  Finally,  the  aviator  selection  research  was  per¬ 
formed  as  part  of  Manpower  and  Personnel  Project  A792,  within  the 
Manpower  and  Personnel  Research  Laboratory. 

The  research  efforts  summarized  in  this  report  respond  to 
several  Memorandums  of  Agreement  (MOAs)  between  the  U.S.  Army 
Research  Institute  for  the  Behavioral  and  Social  Sciences  (ARI) 
and  Aviation  Systems  Command  (AVSCOM) ,  Training  and  Doctrine 
Command  (TRADOC) ,  the  U.S.  Army  Aviation  Center  (USAANVC) ,  and 
Combined  Arms  Test  Activity  (CATA) .  Some  of  the  work  was  in  di¬ 
rect  response  to  letters  from  agencies  in  the  sponsoring  organi¬ 
zation.  Each  project  summary  lists  the  agency,  the  product,  and 
the  outcome. 

This  report  summarizes  research  performed  and  products  de¬ 
veloped  in  all  three  of  the  above  areas  between  9  October  1987 
and  8  October  1988.  Twenty-five  projects  are  described  under  16 
research  area  summaries.  Ten  projects  describe  research  in  sup¬ 
port  of  emerging  systems,  5  present  research  in  support  of  man¬ 
power  and  personnel  programs,  and  10  report  accomplishments  in 
support  of  aviator  training  programs.  In  addition,  6  technical 
advisory  services  are  described. 

This  summary  report  is  intended  to  meet  two  important  objec¬ 
tives.  First,  it  provides  U.S.  Army  weapon  system  managers,  man¬ 
power  and  personnel  planners,  and  training  system  developers  and 
managers  with  a  summary  of  research  progress  and  accomplishments 
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in  their  areas  of  responsibility.  Second,  it  provides  summary 
information  to  behavioral  scientists  who  may  be  working  on  simi¬ 
lar  applied  research  issues,  either  in  the  Department  of  Defense 
or  in  other  governmental,  industrial,  or  university 
organizations. 


Technical  Director 


i  ‘  t 


vi 


V 


HUMAN  FACTORS  RESEARCH  IN  AIRCREW  PERFORMANCE  AND  TRAINING:  1988 
ANNUAL  SUMMARY  REPORT 


EXECUTIVE  SUMMARY 


Requirement: 

Anacapa  Sciences,  Inc.,  has  provided  research  support  to  the 
U.S.  Army  Research  Institute  Aviation  Research  and  Development 
Activity  (ARIARDA)  at  Fort  Rucker,  Alabama,  since  1981.  The 
ARIARDA  program  supports  the  full  range  of  Army  aviation  research 
requirements  with  projects  that  address  emerging  aviation  weapon 
systems,  aviation  manpower  and  personnel  programs,  and  aviator 
training  programs.  This  report  describes  the  25  research  proj¬ 
ects  and  6  technical  advisory  services  conducted  by  Anacapa 
Sciences,  Inc.,  researchers  between  October  1987  and  October  1988 
in  support  of  the  ARIARDA  program.  The  specific  requirements 
that  led  to  each  research  project  are  discussed  in  the  individual 
summaries. 


Procedure : 

There  are  substantial  differences  in  the  methods  used  in  the 
individual  projects  and  in  the  technical  advisory  services.  In 
some  cases,  the  research  approach  was  a  scientific  experiment  in 
which  selected  variables  were  controlled,  manipulated,  and  meas¬ 
ured.  In  other  cases,  the  research  approach  was  a  set  of  anal¬ 
ytical  or  product  development  tasks.  The  specific  research 
methods  used  in  each  project  and  technical  advisory  service  are 
described  in  moderate  detail  in  the  individual  summaries. 


Findings: 

The  research  projects  were  conducted  in  all  three  domains  of 
the  ARIARDA  research  program.  The  25  projects  are  described  in 
16  research  summaries.  Ten  of  the  projects  address  the  emerging 
aviation  weapon  systems  program.  Eight  of  these  projects  address 
the  prediction  of  operator  workload  in  varying  configurations  of 
the  LHX,  AH-64,  UH-60,  MH-60K,  and  MH-47E  helicopters.  The  other 
two  projects  are  concerned  with  flight  symbology  and  integration 
of  maintenance  considerations  during  the  early  design  phases  of 
new  aircraft. 
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Five  of  the  projects  are  in  the  manpower  and  personnel 
research  program:  the  development  of  a  new  aviator  selection 
test,  the  development  of  a  peer  assessment  method,  an  evaluation 
of  the  First  Army  Reserve  aviation  management  method,  an  evalua¬ 
tion  of  Reserve  Component  training  requirements,  and  a  survey  of 
aviation  ammunition  and  gunnery  training  practices  and  require¬ 
ments.  The  remaining  ten  projects  are  part  of  the  aviator  train¬ 
ing  research  program.  Five  of  the  projects  are  concerned  with 
the  evaluation  of  flight  simulator  training  in  operational  Army 
aviation  units.  The  other  five  projects  are  concerned  with  up¬ 
grading  the  basic  map  interpretation  and  terrain  analysis  course 
to  videodisc,  developing  an  AH-64  symbology  training  program, 
surveying  the  research  on  computer-based  instruction,  and  con¬ 
ducting  two  evaluations  of  the  effectiveness  of  aviation  part- 
task  trainers. 

The  six  technical  advisory  services  were  concerned  with 
determining  the  LHX  simulation  requirements,  participating  in  all 
the  Special  Operations  Aircraft  Program  in-progress  reviews  and 
crew  station  working  group  meetings,  developing  an  AH-64  auto¬ 
mated  target  handover  system,  developing  a  multitrack  classifi¬ 
cation  battery,  evaluating  the  relationship  between  handedness 
and  flight  training  performance,  and  developing  software  for  a 
flight line  research  system. 


Utilization  of  Findings: 

The  results  and  recommendations  of  many  of  the  projects  and 
technical  advisory  services  will  aid  in  the  design  of  new  avia¬ 
tion  systems,  in  the  selection  and  management  of  aviation  person¬ 
nel,  and  in  aviation  training  at  the  Aviation  Center  at  Fort 
Rucker,  and  in  world-wide  Army  aviation  units.  This  report 
provides  Army  weapon  systems  managers,  manpower  and  personnel 
planners,  training  system  developers  and  managers,  and  other 
researchers  working  in  related  fields  with  a  summary  of  the 
research  activities  in  their  respective  areas  of  interest. 
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1988  ANNUAL  SUMMARY  REPORT 


INTRODUCTION 

Anacapa  Sciences,  Inc.,  has  provided  collocated  research 
support  to  the  U.S.  Army  Research  Institute  Aviation  Research 
and  Development  Activity  (ARIARDA)  at  Fort  Rucker,  Alabama, 
under  a  series  of  contracts  that  began  1  September  1981.  The 
current  contract  (No.  MDA903-87-C-0523)  requires  the  submis¬ 
sion  of  an  Annual  Summary  Report  of  research  project  activi¬ 
ties.  Project  activities  during  the  first  contract  year  were 
reported  in  Aldrich  and  McAnulty  (1987) .  This  report 
describes  the  Anacapa  research  project  activities  and  achieve¬ 
ments  during  the  period  from  9  October  1987  to  8  October  1988. 
Throughout  the  report,  this  period  is  referred  to  as  the 
current  contract  year. 

During  the  1988  in-progress  review,  some  of  the  Anacapa 
project  activities  were  reorganized  to  (a)  facilitate  project 
accounting,  (b)  distribute  resources  in  accordance  with 
current  and  projected  funding,  and  (c)  increase  productivity. 

Two  major  changes  were  implemented  and  are  reflected  in  the 
organization  of  this  report.  First,  some  of  the  major  long¬ 
term  research  projects  were  designated  as  research  areas  and 
divided  into  discrete  projects  that  will  each  conclude  with  a 
deliverable  product.  As  a  result,  some  of  the  research 
summaries  in  this  report  describe  more  than  one  project. 

Second,  a  new  category  of  activities  called  technical  advisory 
services  was  established  to  provide  research  findings  and  expert 
assistance  to  Army  programs  and  projects  that  are  not  directly 
assigned  to  Anacapa  Sciences. 

The  first  section  of  this  report  contains  summary 
descriptions  of  25  projects  that  Anacapa  personnel  have 
worked  on  during  the  current  contract  year.  Six  technical 
advisory  services  provided  during  the  current  contract  year 
are  summarized  in  the  second  section  of  this  report.  Most  of 
the  project  and  technical  advisory  services  summaries  follow 
the  same  general  format.  Each  summary  begins  with  a  back¬ 
ground  section  that  presents  information  needed  to  understand 
the  requirement  for  the  project.  The  background  may  include 
a  brief  review  of  the  relevant  research  literature  or 
describe  the  critical  events  that  led  to  the  initiation  of 
the  project  or  technical  advisory  service.  Where  appro¬ 
priate,  the  relationship  between  specific  projects  in  a 
research  area  or  between  an  Anacapa  research  project  and  a 
technical  advisory  service  is  discussed. 
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When  the  need  for  the  research  cannot  be  clearly 
inferred  from  the  background  information,  a  statement  of  need 
or  definition  of  the  research  problem  is  presented.  This  is 
followed  by  a  concise  statement  of  the  project  or  research 
area  objectives.  Next,  the  research  approach  section 
presents  a  description  of  the  activities  that  were  planned  to 
accomplish  the  research  objectives.  For  some  projects,  the 
research  approach  is  a  scientific  experiment  in  which 
selected  variables  are  controlled,  manipulated,  and  measured. 
For  other  projects,  the  research  approach  is  a  set  of 
analytical  or  product  development  tasks. 

In  the  project  summaries,  the  research  approach  is 
usually  followed  by  one  or  more  sections  that  present  the 
research  findings  or,  in  the  case  of  product  development 
efforts,  a  summary  description  of  the  research  products.  In 
the  technical  advisory  service  summaries,  the  research 
approach  is  usually  followed  by  a  description  of  the  services 
provided  by  Anacapa  personnel.  The  final  section  of  each 
summary,  entitled  "Project  Status,"  describes  the  work 
accomplished  during  the  contract  year  and  the  work  projected, 
if  any.  This  section  also  identifies  delivered  products  and 
reports  and,  where  possible,  presents  the  current  project 
milestones . 

Anacapa  personnel  also  provided  temporary  research, 
technical,  administrative,  and  logistical  support  on  other 
projects  that  are  the  primary  responsibility  of  ARIARDA 
personnel  and  are,  consequently,  not  summarized  in  this 
report.  It  is  also  important  to  note  that  the  projects 
summarized  in  this  report  represent  only  a  portion  of 
ARIARDA' s  research  program.  Numerous  other  projects  are 
being  conducted  either  in-house  by  ARIARDA  personnel  or  under 
other  contracts. 

The  project  summaries  are  presented  in  three  content 
categories  that  reflect  the  research  domains  at  ARIARDA. 

This  organization  is  intended  to  assist  the  reader  in 
locating  a  specific  project  summary  within  a  research  domain 
or  to  find  summaries  that  are  closely  related  in  terms  of 
content . 

The  first  six  summaries  describe  ten  projects  in 
emerging  aviation  systems  design.  The  next  five  summaries 
present  projects  in  manpower  and  personnel  research.  The 
last  five  summaries  describe  ten  aviator  training  research 
projects.  The  number  uf  projects  assigned  to  the  three 
categories  is  not  necessarily  in  proportion  to  the  emphasis 
placed  on  each  research  domain. 
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The  project  summaries  are  followed  by  descriptions  of 
the  six  technical  advisory  services.  Technical  advisory 
services  were  provided  to  support  (a)  aviation  system 
research  and  development  programs  being  managed  by  Aviation 
Systems  Command  managers  and  engineers,  and  (b)  aviation 
projects  being  conducted  by  ARIARDA  scientists.  The  first 
three  summaries  describe  technical  advisory  services  provided 
in  support  of  emerging  aviation  systems  design.  The  next  two 
summaries  describe  technical  advisory  services  provided  in 
support  of  manpower  and  personnel  research.  The  sixth 
summary  describes  a  technical  advisory  service  provided  in 
support  of  an  ARIARDA  training  research  project. 

Although  each  summary  identifies  the  project  director (s) 
or  technical  advisor (s),  the  Anacapa  approach  to  research 
employs  a  team  concept.  This  approach  provides  the  optimum 
utilization  of  each  scientific  staff  member's  skills  and 
ensures  coordination  among  closely  related  projects.  The 
scientific  staff  members  are  supported  by  an  exceptionally 
efficient  administrative  and  technical  staff.  All  of  the 
research  effort  is  closely  coordinated  with  ARIARDA 
personnel . 


Rsf£,r.£JQ££ 

Aldrich,  T.  B.,  &  McAnulty,  D.  M.  (1987).  Human  factors 

research  in  aircrew  performance  and  training:  Annual 
summary  report  (Technical  Report  ASI690-304-87) .  Fort 
Rucker,  AL:  Anacapa  Sciences,  Inc. 
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VALIDATION  OF  THE  LHX  WORKLOAD  PREDICTION  MODEL 
Mr.  Theodore  B.  Aldrich,  Project  Director 


Background 

Models  that  predict  operator  workload  can  be  useful 
tools  for  human  factors  engineers  who  are  addressing  human 
capabilities  and  limitations  during  the  design  of  advanced 
technology  weapon  systems.  Accordingly,  Anacapa  Sciences 
researchers,  under  contract  to  the  U.S.  Army  Research 
Institute  Aviation  Research  and  Development  Activity, 
developed  a  workload  prediction  methodology  and  produced  one- 
and  two-crewmember  models  for  predicting  aviator  workload  in 
advance  of  aircraft  system  design.  The  workload  prediction 
methodology  operationally  defines  workload  in  terms  of 
attentional  demand  and  predicts  workload  associated  with 
task-level  performance.  The  Anacapa  researchers  applied  the 
workload  prediction  models  during  the  conceptual  design  phase 
of  a  proposed  multipurpose,  lightweight  helicopter  designated 
the  LHX  (Aldrich,  Szabo,  &  Craddock,  1986) . 


Neither  the  workload  predictors  used  to  develop  the 
models  nor  the  workload  predictions  yielded  by  the  models 
have  been  validated.  Workload  model  predictors  that  require 
validation  include  the: 

•  workload  ratings  assigned  to  each  task, 

•  total  workload  estimates  for  concurrent  tasks, 

•  estimated  time  required  to  perform  each  task, 

•  temporal  relationships  among  tasks,  and 

•  sequential  relationships  among  tasks. 

Specific  predictions  yielded  by  the  models  that  require 
validation  include  the  four  indexes  of  excessive  workload 
(Aldrich,  Craddock,  &  McCracken,  1984)  listed  below: 

•  component  overloads, 

•  overload  conditions, 

•  overload  density,  and 

•  subsystem  overloads. 


Project  Objectives 

This  project  is  divided  into  three  phases.  The  objec¬ 
tives  of  Phase  1  are  to  evaluate  the  reliability  of  (a)  the 
scales  used  to  rate  the  workload  components  of  each  operator 
task  identified  during  the  LHX  workload  analyses  and  (b)  the 
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workload  predictors  used  in  developing  the  LHX  workload 
prediction  model.  The  objective  of  Phase  2  is  to  obtain 
validation  data  through  part-mission  and  full-mission  flight 
simulation  research.  The  objective  of  Phase  3  is  to  refine 
the  workload  prediction  model  on  the  basis  of  validation 
research  results. 


Methodology 

A  research  plan  designed  to  meet  the  above  objectives 
was  produced  early  in  the  first  contract  year  (Aldrich  & 
Szabo,  1986) .  The  research  plan  provides  detailed  descrip¬ 
tions  of  18  tasks  required  to  accomplish  the  three  phases  of 
the  validation  research.  A  summary  of  the  research  method¬ 
ology  for  each  of  the  three  phases  is  described  below. 


Phase  1 

In  Phase  1,  two  surveys  will  be  administered  to  human 
factors  scientists  who  are  familiar  with  workload  research. 

In  the  first  survey,  all  possible  pairs  of  the  verbal  anchors 
from  each  of  the  workload  component  rating  scales  are  pre¬ 
sented  to  the  workload  subject  matter  experts  (SMEs) .  The 
SMEs  will  select  the  anchor  in  each  pair  that  requires  the 
greatest  attentional  demand.  The  results  of  this  survey  will 
be  used  to  assess  the  interrater  reliability  of  the  scale 
anchors.  The  data  will  also  be  used  to  produce  equal 
interval  scales  (e.g.,  Engen,  1971)  to  replace  the  ordinal 
scales  that  were  used  in  the  original  workload  analysis  . 

The  second  survey  asks  the  same  SMEs  to  use  the  workload 
component  scales  to  rate  the  short  descriptors  of  visual, 
auditory,  cognitive,  and  psychomotor  components  of  workload 
for  each  task  in  the  model.  Correlational  techniques  will  be 
used  to  evaluate  the  interrater  reliability  of  the  workload 
rat ings . 


Phase  ,2 

In  Phase  2,  part-mission  and  full-mission  simulation 
experiments  will  be  conducted  to  validate  the  workload  esti¬ 
mates.  For  the  part-mission  simulation,  mini-scenarios  will 
be  generated  by  selecting  concurrent  and  sequential  tasks 
from  the  mission/task  analysis.  For  the  full-mission  simu¬ 
lation,  a  composite  mission  scenario  will  be  developed  by 
selecting  segments  from  the  mission/task  analysis. 
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The  part-mission  simulation  experiments  will  be 
conducted  using  a  repeated  measures  experimental  design  in 
which  each  subject  will  fly  the  mini-scenarios  multiple 
times.  The  results  will  be  analyzed  by  correlating  the 
workload  predictors  and  the  measures  of  the  operators' 
performance  on  the  concurrent  and  sequential  tasks.  The 
correlation  coefficients  will  indicate  how  accurately  the 
workload  predictors  forecast  excessive  workload  at  the  task 
level.  To  assess  the  validity  of  the  time  estimates  used  in 
the  model,  the  time  required  to  perform  the  various  tasks  in 
the  mini-scenarios  will  be  compared  with  the  times  estimated 
during  the  task  analysis.  The  procedural  relationships  among 
the  tasks  will  be  evaluated  by  noting  the  subjects'  ability 
to  progress  through  the  mini-scenarios  following  the  sequence 
of  tasks  specified  by  the  model. 

During  the  full-mission  simulation  experiments,  each 
trial  will  start  at  the  beginning  of  a  composite  scenario  and 
continue  without  interruption  to  the  end.  All  the  part- 
mission  simulation  data  analyses  will  be  conducted  on  the 
full-mission  simulation  data.  In  addition,  an  analysis  will 
be  performed  to  assess  the  effects  of  inserting  secondary 
tasks  into  the  composite  mission  scenario. 

The  final  task  in  Phase  2  will  be  to  compare  the  results 
from  the  part-mission  simulation  research  with  results  from 
the  full-mission  simulation  research.  The  findings  from  this 
comparison  will  be  used  to  determine  if  excessive  workload 
results  from  the  cumulative  effects  of  high  workload  over  the 
longer  times  in  the  composite  mission  scenario. 


Phase  3 

In  Phase  3,  refinements  will  be  made  to  the  workload 
prediction  model  on  the  basis  of  the  results  from  the  first 
two  phases.  First,  the  workload  component  rating  scales  will 
be  converted  from  ordinal  to  interval  scales.  Second, 
refinements  will  be  made  to  the  workload  model  algorithms  to 
reflect  the  empirical  results  of  the  part-mission  and  full- 
mission  simulation  experiments. 


Results 


Phase  1 

The  first  survey  has  been  conducted.  A  survey  instru¬ 
ment  was  developed  that  presented  all  possible  pairs  of  the 
verbal  anchors  from  each  of  the  four  workload  component 
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rating  scales.  The  instrument  was  mailed  to  71  SMEs  who 
selected  the  verbal  anchor  in  each  pair  judged  to  require  the 
greatest  attentional  demand.  The  data  from  38  completed 
survey  instruments  were  used  to  develop  each  rater's  rank 
order  judgment  of  the  verbal  anchors.  Kendall's  Coefficient 
of  Concordance  (Siegal,  1956)  was  used  to  assess  the  degree 
of  agreement  among  the  SMEs.  The  Coefficients  of  Concordance 
for  the  four  scales  are  as  follow: 

•  Visual  -  .39, 

•  Auditory  -  .46, 

•  Cognitive  -  .69,  and 

•  Psychomotor  -  .47. 

All  of  the  Coefficients  of  Concordance  are  significant  at  the 
.001  level  and  indicate  a  moderate  degree  of  consensus  among 
the  SMEs. 


Phase  2 


In  October  1987,  a  new  Crew  Station  Research  and 
Development  Facility  (CSRDF)  located  at  the  Army's 
Aerof lightdynamics  Laboratory,  NASA  Ames,  Moffett  Field, 
California,  was  selected  as  the  most  appropriate  site  for 
conducting  the  validation  research.  CSRDF  researchers 
originally  planned  to  support  the  LHX  validation  research 
with  part-mission  simulation  research  scheduled  during 
October  1988,  January  1989,  and  April  1989  and  full-mission 
simulation  research  scheduled  from  July  through  September  of 
1989. 


In  April  1988,  CSRDF  planners  postponed  the  simulation 
research  from  October  1988  until  October  1989  because  of 
delays  in  acquiring  a  new  Compuscene  IV  visual  system  and 
because  of  changes  in  research  priorities.  CSRDF  simulation 
planners  also  stated  that  the  initial  LHX  validation  research 
should  be  full-mission  simulation  with  two  crewmembers.  This 
differs  from  the  project  research  plan  that  begins  with  the 
part-mission,  one-crewmember  simulation. 


Phase  3 


The  data  from  the  pair  comparison  survey  were  used  to 
construct  interval  values  for  the  workload  component  scales. 
The  workload  prediction  model  was  refined  by  replacing  the 
original  ordinal  values  for  each  task  with  the  interval 
values.  The  one-crewmember  workload  model  was  exercised  with 
the  new  interval  scale  values  to  produce  estimates  of  work¬ 
load  for  each  of  the  29  mission  segments  in  the  model. 
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Table  1  presents  a  comparison  of  the  workload  predicted  by 
the  interval  scale  values  and  the  ordinal  scale  values.  The 
results  presented  are  total  values  for  all  29  segments  in  the 
model.  Although  there  are  some  differences,  interval  scale 
values  closely  correspond  to  the  results  produced  by  the 
ordinal  scale  values. 


Table  1 

Workload  Predicted  by  the  Ordinal  and  Interval  Scale  values 


Overload 

Conditions 

Component 

Overloads 

Overload 

Densitv 

Visual 

Cognitive 

Psychomotor 

Ordinal  Values 

263 

81 

52 

203 

.206 

Interval  Values 

292 

94 

92 

4.  «-  * 

.202 

Pr..c>..is,.c.t--Stat,us. 


Phase  1 

The  final  step  in  Phase  1  is  to  design  and  administer  a 
second  survey  instrument.  The  instrument  will  request  that 
SMEs  rate  verbal  descriptors  of  LHX  tasks  using  the  workload 
component  scales.  Data  from  the  survey  will  be  used  to 
assess  the  interrater  reliability  of  the  scales. 


Phase  2 

CSRDF  plans  have  forced  a  delay  in  the  work  required  to 
accomplish  Phase  2.  Anacapa  still  plans  to  use  flight  simu¬ 
lation  to  validate  the  LHX  workload  prediction  model,  but  the 
plans  depend  upon  access  to  a  high-fidelity,  high-technology 
flight  simulator. 


Phase  3 

Further  comparisons  will  be  made  between  the  interval 
and  ordinal  scale  values  with  the  two-crewmember  mo^01  and 
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with  various  automation  options.  Additional  refinements  may 
be  identified  when  the  model  is  exercised  to  produce  workload 
predictions  using  the  new  scale  values. 

Additional  refinements  will  be  made  to  the  workload 
prediction  model  when  the  flight  simulation  research  is 
conducted  and  analyzed.  Refinements  that  may  be  introduced 
include  changes  to  the  workload  estimates,  time  estimates, 
and  decision  rules.  New  workload  predictions  will  be 
produced  following  each  improvement  in  the  model. 
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DEVELOPMENT  OF  A  TASK  ANALYSIS/ WORKLOAD 
(TAWL)  SOFTWARE  SYSTEM 

Ms.  Cassandra  Hocutt,  Project  Director 


Background 

Anacapa  Sciences  researchers,  under  contract  to  the  Army 
Research  Institute  Aviation  Research  and  Development  Activity 
(ARIARDA) ,  Fort  Rucker,  Alabama,  have  developed  a  methodology 
for  predicting  operator  workload  during  the  conceptual  phase 
of  new  weapon  system  development .  The  methodology  was  first 
applied  to  the  Army's  Light  Helicopter  Family  (LHX)  aircraft 
(McCracken  &  Aldrich  1984;  Aldrich,  Craddock,  &  McCracken, 
1984;  Aldrich,  Szabo,  &  Craddock,  1986). 

Subsequently,  Anacapa  personnel  refined  the  mission/ 
task/workload  analysis  methodology  and  produced  operator 
workload  prediction  models  for  the  AH-64A  (Szabo  &  Bierbaum, 
1986) ,  the  UH-60  (Bierbaum,  Szabo,  &  Aldrich,  1987) ,  and  the 
CH-47  aircraft.  Each  of  the  original  workload  prediction 
models  was  programmed  in  FORTRAN  77  on  a  Perkin-Elmer  3210. 
Currently,  Anacapa  researchers  are  developing  workload 
prediction  models  for  the  MH-60K  and  the  MH-47E  aircraft. 

The  Anacapa  project  director  is  developing  a  software  system, 
called  the  Task  Analysis/Workload  (TAWL) ,  that  is  capable  of 
exercising  any  of  the  models  developed  with  the  refined 
methodology . 


Meed 

Programs  for  the  LHX,  AH-64A,  and  UH-60  workload  predic¬ 
tion  models  incorporate  the  model  decision  rules  into  the 
actual  program  code.  Time-consuming  recompilations  of  the 
programs  are  required  to  incorporate  even  minor  changes  in 
the  models.  A  software  system  is  required  that  reduces  the 
development  time  for  implementing  changes  to  existing  models 
or  creating  new  models. 


Project  Objectives 

The  primary  objective  of  this  project  is  to  develop  a 
software  system  that  (a)  can  incorporate  model  changes  with¬ 
out  rewriting  and  recompiling  the  software,  and  (b)  is  power¬ 
ful  and  flexible  enough  to  exercise  any  of  the  workload 
prediction  models  developed  with  the  refined  mission/task/ 
workload  analysis  methodology.  In  addition,  the  software 
system  should  be: 
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•  easy  to  use, 

•  portable,  and 

•  easy  to  modify  in  accordance  with  changing  require¬ 
ments  created  by  the  development  of  a  new  workload 
prediction  model. 


Approach /Methodology 

The  TAWL  software  system  uses  data  files  to  store  all 
model  information  peculiar  to  a  specific  aircraft.  This 
design  approach  enables  the  programmer: 

•  to  change  an  existing  model's  execution  by  changing 
its  data  files,  thereby  eliminating  the  need  to 
rewrite  and  recompile  the  program  to  incorporate  the 
changes;  and 

•  to  implement  new  computer  models  developed  with  the 
refined  methodology  merely  by  creating  a  new  set  of 
data  files. 

Although  the  technique  of  using  data  files  to  store  all 
model  information  resolved  the  problems  described  above,  the 
data  entry  of  model  information  using  an  editor  is  time 
consuming  and  subject  to  errors.  A  data  base  management 
system  with  specialized  routines  was  designed  for  entering 
and  updating  all  of  the  data  used  in  the  workload  prediction 
models.  Each  specialized  routine  features  customized  error 
checks  to  help  ensure  the  validity  of  the  data  files.  The 
most  critical  data  files  are  protected  by  automatic  backup 
procedures . 

A  simple  and  consistent  user-interface  was  developed  to 
produce  a  system  that  is  easy  to  use.  The  software  system 
was  developed  for  use  on  an  IBM  AT  compatible  computer  to 
meet  the  objective  of  portability.  Turbo  Pascal  4.0  was 
selected  as  the  development  language  to  meet  the  objective  of 
easy  modification.  Turbo  Pascal  4.0  has  the  following 
features . 

•  The  control  and  data  structures  can  support  the  design 
complexity  required  to  exercise  the  workload 
prediction  models. 

•  The  control  and  data  structures  enable  the  programmer 
to  produce  code  that  is  clean,  clear,  and  easy  to 
modify . 

•  The  excellent  development  environment  includes  special 
debugging  features  and  a  built-in  editor  designed  for 
program-style  documents. 
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The  rigid  syntax  enables  the  compiler  to  identify 
errors  that  require  time-consuming  debugging  in  other 
languages . 


Project  Status 


Work  Completed 

The  first  version  of  the  TAWL  software  system  has  been 
released  and  is  currently  in  use.  It  was  used  to  exercise 
the  CH-47  model  and  is  being  used  to  build  a  data  base  for 
the  revision  of  the  AH-64A  model.  A  data  base  for  the  UH-60 
model  was  created  and  validated  using  TAWL ' s  current  version 
Finally,  work  has  begun  on  creating  TAWL  data  files  for  the 
MH-47E  and  the  MH-60K  helicopters. 

A  copy  of  the  software  system,  TAWL's  UH-60  data  files, 
and  a  draft  user's  guide  have  been  submitted  for  testing  and 
review  by  personnel  at  the  ARI  AVSCOM  Element  office,  St. 
Louis,  Missouri. 


Work  Projected 

Work  will  continue  on  the  implementation  of  the  TAWL 
system  for  the  AH-64A,  MH-47E,  and  MH-60K  models.  Other 
models  or  changes  to  existing  models  will  be  implemented  as 
required.  A  limited  directory  utility  with  a  graphic  inter¬ 
face  is  scheduled  for  the  TAWL  software  system  as  a  design 
improvement.  The  utility  will  be  developed  as  a  separate 
program  and  will  be  incorporated  into  the  software  system 
after  the  directory  and  file  routines  have  been  tested.  The 
directory  utility  will  add  the  following  improvements  to  the 
software  system: 

•  the  ability  to  build  and  work  with  a  data  base  in  any 
directory  (TAWL's  current  version  is  limited  to  the 
root  directory  or  a  first-level  directory) ; 

•  the  ability  to  add,  delete,  and  rename  directories; 

•  the  ability  to  rename,  copy,  and  move  files;  and 

•  an  improved  interface  for  all  file  and  directory 
routines . 

Other  improvements  will  be  scheduled  as  needs  arise.  A 
user's  guide  for  the  TAWL  software  system  also  will  be 
produced . 
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UH/MH-60  AND  CH/MH-47  TASK/WORKLOAD  ANALYSES 
Mr.  Carl  R.  Bierbaum,  Project  Director 


Background 

The  U.S.  Army  presently  utilizes  CH-47D  and  UH-60A 
aircraft  to  support  Special  Operations  Forces  (SOF) . 

Although  the  SOF  aircraft  have  special  systems  to  assist 
pilots  in  performing  the  special  operations  missions,  the 
Special  Operations  Aircraft  (SOA)  Program  Manager's  (PM) 
Office  at  the  Army's  Aviation  Systems  Command  has  been  tasked 
to  develop  an  MH-60K  and  an  MH-47E  aircraft  for  the  SOF.  The 
SOA  will  consist  of  existing  CH-47D  and  UH-60A  airframes  with 
new  integrated  cockpits.  Specifically,  four  multifunction 
display  (MFD)  units  will  replace  the  existing  instrument  and 
gauge  configurations  in  both  the  CH-47D  and  UH-60A  aircraft. 

Anacapa  Sciences  researchers,  under  contract  to  the  Army 
Research  Institute  Aviation  Research  and  Development  Activity 
(ARIARDA) ,  have  developed  a  methodology  tor  predicting 
operator  workload  during  system  design.  Initially,  the 
workload  prediction  methodology  was  developed  and  applied  to 
the  design  of  the  Army's  light  helicopter  family  (LHX)  air¬ 
craft  (Aldrich,  Szabo,  &  Craddock,  1986) .  Later,  the  LHX 
workload  prediction  methodology  was  refined  and  used  to 
predict  workload  encountered  by  operators  of  the  following 
helicopter  systems: 

•  the  AH-64A  Apache  (Szabo  &  Bierbaum,  1986) ,  and 

•  the  UH-60A  Black  Hawk  (Bierbaum,  Szabo,  &  Aldrich, 

1987)  . 

The  high  technology  modifications  proposed  for  the  SOA 
cockpits  may  increase  workload  by  placing  additional  mental 
and  visual  demands  on  crewmembers  as  they  attempt  to  inter¬ 
pret  the  many  MFD  options.  Therefore,  the  SOA  PM  Office 
requested  that  ARIARDA  produce  the  mission/task  analyses  and 
workload  predictions  for  the  MH-60K  and  MH-47E  aircraft. 


Research  Objectives 

The  overall  objective  of  this  research  area  is  to 
determine  the  impact  that  high  technology  modifications  are 
likely  to  have  on  the  workload  of  MH-60K  and  MH-47E  crew¬ 
members.  Specifically,  the  research  is  designed  to: 

•  provide  a  mission/task  analysis  of  the  UH-60A  and  the 
MH-60K, 
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•  determine  the  impact  that  the  proposed  MH-60K  modifi¬ 
cations  will  have  on  crew  workload, 

•  provide  a  mission/task  analysis  of  the  CH-47D  and  the 
MH-47E,  and 

•  determine  the  impact  that  the  proposed  MH-47E  modifi¬ 
cations  will  have  on  crew  workload. 


Method 

The  approach  selected  for  meeting  the  research  objec¬ 
tives  is  a  refinement  of  the  workload  prediction  methodology 
employed  during  the  LHX  research  (Aldrich,  Szabo,  &  Craddock, 
1986)  and  the  AH-64A  research  (Szabo  &  Bierbaum,  1986) .  The 
research  is  divided  into  four  projects. 

•  In  previous  research,  a  baseline  mission/task  analysis 
of  crew  workload  was  conducted  for  the  UH-60A 
helicopter,  and  a  computer  model  of  UH-60A  workload 
was  developed. 

•  Currently,  a  baseline  mission/task  analysis  of  crew 
workload  is  being  conducted  for  the  CH-47D  helicopter, 
and  a  computer  model  of  CH-47D  workload  is  being 
developed . 

•  In  a  related  research  project,  the  UH-60A  workload 
prediction  model  will  be  exercised  to  predict  the 
impact  that  various  MH-60K  design  modifications  are 
likely  to  have  on  crew  workload. 

•  In  another  related  research  project,  the  CH-47D  work¬ 
load  prediction  model  will  be  exercised  to  predict  the 
impact  that  MH-47E  design  modifications  are  likely  to 
have  on  crew  workload. 


Status 

The  development  of  the  UH-60A  workload  model  was  com¬ 
pleted  during  the  first  contract  year  and  was  reported  in  the 
1987  Annual  Summary  Report  (Aldrich  &  McAnulty,  1987)  and  in 
a  research  product  by  Bierbaum  et  al.  (1987) . 


CH-47D  Task  Analysis  and  Workload  Prediction  Model 

The  development  of  the  CH-47D  workload  model  was  com¬ 
pleted  during  the  current  contract  year.  During  the  next 
contract  year,  a  technical  report  will  be  prepared  that 


16 


describes  the  task  analysis  and  the  development  of  the  CH-47D 
workload  prediction  model.  The  completed  project  tasks  are 
described  briefly  below. 

Develop  a  mission  scenario.  This  section  describes  the 
mission  scenario  that  was  developed  in  the  CH-47D  analysis. 
The  CH-47D  mission  begins  in  an  assembly  area  (AA) .  The 
pilot  flies  contour  flight  from  the  AA  to  a  pick-up  zone 
(PZ) ,  where  cargo  and/or  troops  are  assembled  for  pick  up. 
After  completing  loading  operations,  the  pilot  flies  nap-of- 
the-earth  (NOE)  to  the  landing  zone  (LZ)  to  insert  the  combat 
troops  or  deliver  the  cargo.  After  completing  the  delivery, 
the  pilot  flies  NOE  to  the  PZ  for  another  load.  This  pattern 
is  continued  until  refueling  is  required.  The  pilot  then 
flies  NOE  from  the  LZ  to  the  forward  area  arming  and  refuel¬ 
ing  point  (FARP) ,  where  refueling  operations  are  conducted. 
Upon  completion  of  the  FARP  operations,  the  crew  returns  to 
the  PZ  for  continuation  of  the  mission.  When  the  mission  is 
completed,  the  pilot  flies  contour  back  to  the  AA.  Preflight 
and  postflight  activities  are  not  included  in  the  analysis. 

Divide  the  mission  scenario  into  ..mission  phases.  The 
mission  scenario  was  divided  into  the  following  nine  phases: 

•  Departure  (AA) , 

•  Enroute  (AA-PZ), 

•  Departure  (PZ)  , 

•  Enroute  (PZ-LZ), 

•  Departure  (LZ)  , 

•  Enroute  (LZ-PZ)  or  (LZ-FARP), 

•  FARP  Operations, 

•  Enroute  (FARP-PZ) ,  and 

•  Enroute  (PZ-AA) . 

Divide  the,  mission  phases  into  segments.  The  nine 
mission  phases  were  divided  into  37  unique  segments.  Some 
segments  are  used  in  more  than  one  of  the  mission  phases. 

The  number  of  segments  in  each  mission  phase  is  listed  below: 

•  Departure  (AA-PZ)  -  4, 

•  Enroute  (AA-PZ)  -  10, 

•  Departure  (PZ)  -  7, 

•  Enroute  (PZ-LZ)  -  12, 

•  Departure  (LZ)  -  3, 

•  Enroute  (LZ-PZ)  or  (LZ-FARP)  -  10, 

•  FARP  Operations  -  5, 

•  Enroute  (FARP-PZ)  -  10,  and 

•  Enroute  (PZ-AA)  -  10. 
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Divide  the  segments  into  functions.  Each  of  the  37 
unique  segments  was  divided  into  functions.  A  total  of  66 
unique  functions  was  identified,  alphabetized,  and  assigned  a 
numerical  identification  code. 

Identify  the  specific  tasks  for  each  function.  Each  of 
the  66  functions  was  divided  into  tasks.  A  total  of  154 
unique  tasks  was  identified,  alphabetized  by  object,  and 
assigned  a  numerical  identification  code. 

Identify  the  subsystems  associated  with  _each^.t_ask  . 
Seventeen  aircraft  subsystems  were  identified  on  the  CH-47D. 
The  subsystems  are  divided  into  the  five  major  categories 
listed  below: 

•  Engine, 

•  Flight  Control, 

•  Navigation, 

•  Utility,  and 

•  Visual 

Estimate  the  workload  for  each  task.  A  short  verbal 
description  of  each  of  the  workload  components  was  written 
for  each  task.  The  descriptors  were  then  compared  to  verbal 
anchors  contained  in  7-point  rating  scales  designed  for 
rating  each  of  the  five  workload  components.  The  verbal 
descriptions  and  ratings  were  reviewed  by  CH-47D  subject 
matter  experts  (SMEs) . 

Estimate  the  time  required  to  perform  each  task.  The 
CH-47D  simulator  was  not  available  for  empirically  recording 
task  times  during  this  project.  Therefore,  the  time  assigned 
to  each  task  was  developed  from  SME  estimates.  The  SMEs 
estimated  the  times  based  upon  their  experience  in  performing 
the  tasks . 

Develop  and  exercise  the  CH-47D  model.  The  specific 
tasks  identified  during  the  analysis  were  entered  into  data 
files  for  the  workload  prediction  model.  The  researchers 
developed  decision  rules  required  to  combine  specific  tasks 
into  functions  and  to  combine  the  functions  into  segments. 

All  decision  rules  were  then  programmed  and  the  workload 
prediction  model  was  exercised  to  provide  estimates  of 
workload  at  each  half-second  interval  for  each  of  the  37 
segments  in  the  analysis. 


MH-60K  Task  Analysis  and  Workload  Prediction  Model 

During  the  current  contract  year,  work  progressed  on  the 
MH-60K  task  analysis  and  on  the  development  of  an  MH-60K 
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workload  prediction  model.  The  completed  project  tasks  are 
described  briefly  below. 

Dpyplnp  a  mission  scenario.  This  section  describes  the 
mission  scenario  that  was  developed  for  the  MH-60K  analysis. 
The  MH-60K  mission  begins  with  a  departure  from  a  base  site. 
The  pilot  flies  contour  from  the  base  to  a  rendezvous  point, 
where  air-to-air  refueling  is  accomplished.  After  refueling, 
the  pilot  flies  NOE  from  the  rendezvous  point  to  the  LZ .  The 
pilot  then  flies  back  to  a  rendezvous  point  for  refueling  and 
continues  to  the  base.  The  complete  mission  is  conducted  at 
night  with  night  vision  goggles.  Preflight  and  postflight 
activities  are  not  included  in  the  analysis. 

Divide  the  mission  scenario  into  mission  phases .  The 
mission  scenario  was  divided  into  five  phases  as  follow: 

•  Departure  (Base) , 

•  Enroute  (Base-Rendezvous) , 

•  Enroute  (Rendezvous-LZ ) , 

•  Enroute  (LZ-Rendezvous) ,  and 

•  Enroute  (Rendezvous-Base) . 

Divide  the  mission  phases  into  segments.  The  five 
mission  phases  were  divided  into  14  unique  segments.  Some 
segments  are  used  in  more  than  one  of  the  phases.  The  number 
of  segments  in  each  phase  is  listed  below: 

•  Departure  (Base)  -  2, 

•  Enroute  (Base-Rendezvous)  -  4, 

•  Enroute  (Rendezvous-LZ)  -  4, 

•  Enroute  (LZ-Rendezvous)  -  4,  and 

•  Enroute  (Rendezvous-Base)  -  3. 

Divide  the  segments  into  functions.  Each  of  the  14 
unique  segments  was  divided  into  functions.  A  total  of  53 
unique  functions  was  identified,  alphabetized,  and  assigned  a 
numerical  identification  code. 

The  following  research  activities  are  required  to 
complete  the  MH-60K  task  analysis  and  the  development  of  the 
MH-60K  workload  prediction  model: 

•  identify  the  specific  tasks  for  each  function, 

•  identify  the  subsystems  associated  with  each  task, 

•  estimate  the  workload  for  each  task, 

•  estimate  the  time  required  to  perform  each  task, 

•  develop  decision  rules  for  the  functions,  and 

•  develop  decision  rules  for  the  segments. 

These  tasks  will  be  completed  during  the  next  contract  year. 
Upon  completion  of  the  tasks,  all  data  will  be  entered  into 
data  files,  the  MH-60K  workload  prediction  model  will  be 
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developed,  and  the  model  will  be  exercised  to  predict  the 
impact  that  the  design  modifications  are  likely  to  have  on 
the  MH-60K  crew  workload.  A  research  report  will  be  prepared 
to  describe  the  task  analysis,  the  development  of  the  MH-60K 
workload  prediction  model,  and  the  results  from  exercising 
the  model . 


MH-47E  Task  Analysis  and  Workload  Prediction  Model 

During  the  current  contract  year,  work  also  progressed 
on  the  MH-47E  task  analysis  and  on  the  development  of  an  MH- 
47E  workload  prediction  model.  The  mission  scenario,  mission 
phases,  segments,  and  functions  for  the  MH-47E  are  the  same 
as  those  identified  for  the  MH-60K. 

The  following  research  activities  are  required  to  com¬ 
plete  the  MH-47E  task  analysis  and  the  development  of  the  MH- 
47E  workload  prediction  model: 

•  identify  the  specific  tasks  for  each  function, 

•  identify  the  subsystems  associated  with  each  task, 

•  estimate  the  workload  for  each  task, 

•  estimate  the  time  required  to  perform  each  task, 

•  develop  decision  rules  for  the  functions,  and 

•  develop  decision  rules  for  the  segments. 

Upon  completion  of  these  tasks,  all  data  will  be  entered  into 
data  files,  the  MH-47E  workload  prediction  model  will  be 
developed,  and  the  model  will  be  exercised  to  predict  the 
impact  that  the  design  modifications  are  likely  to  have  on 
the  MH-47E  crew  workload.  A  research  report  will  be  prepared 
to  describe  the  task  analysis,  the  development  of  the  MH-47E 
workload  prediction  model,  and  the  results  from  exercising 
the  model . 
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DEVELOPMENT  AND  VALIDATION  OF  THE  AH-64A 
WORKLOAD  PREDICTION  MODEL 


Dr.  David  B.  Hamilton  and  Dr.  Sandra  M.  Szabo, 
Project  Directors 


Background 

The  Army's  Air/Land  Battle  2000  scenario  presents  a 
high-threat  environment  that  will  place  heavy  workload 
demands  on  combat  helicopter  operators.  To  increase  mission 
effectiveness  in  this  environment,  the  latest  Army  heli¬ 
copters  have  been  equipped  with  advanced  technology.  This 
technology  includes  electronic  sensor  capabilities  that 
increase  both  the  amount  and  fidelity  of  information  avail¬ 
able  to  the  operators. 

The  AH-64A  attack  helicopter  is  equipped  with  the  most 
advanced  technology  of  any  helicopter  in  the  U.S.  Army 
inventory.  It  is  the  first  Army  aircraft  equipped  with 
flight  and  weapon  systems  that  allow  missions  to  be  conducted 
at  night  and  under  adverse  weather  conditions.  However,  the 
increased  mission  capabilities  of  the  aircraft  have  dramati¬ 
cally  increased  the  amount  of  information  that  the  operators 
must  process.  The  AH-64A  is  equipped  with  automated  flight 
and  combat  (acquisition,  targeting,  and  engagement)  tech¬ 
nology  that  is  intended  to  reduce  crew  workload.  In  some 
instances,  however,  the  tasks  required  to  use  the  technology 
have  either  increased  workload  or  simply  changed  the  nature 
of  the  task  without  decreasing  workload.  High  workload,  in 
turn,  reduces  mission  effectiveness,  increases  system  manning 
requirements,  and  increases  the  training  necessary  for 
acquiring  and  maintaining  flight  proficiency. 

One  of  the  reasons  that  technology  has  failed  to  reduce 
operator  workload  in  current  Army  aircraft  is  that  human 
factors  concepts  were  not  adequately  considered  during  the 
early  stages  of  system  design.  In  the  past,  no  methodology 
existed  for  assessing  the  workload  demands  of  emerging 
aviation/weapon  systems  prior  to  their  development.  Recently, 
however,  researchers  from  the  U.S.  Army  Research  Institute 
Aviation  Research  and  Development  Activity  ( ARIARDA)  and 
Anacapa  Sciences  developed  a  methodology  for  predicting  the 
workload  demands  placed  on  the  crewmembers  by  the  advanced 
technology  proposed  for  the  light  helicopter  family  (LHX) 
aircraft.  The  methodology  for  predicting  LHX  workload  in 
advance  of  system  development  can  be  extended  to  existing 
weapon  systems . 


The  methodology's  multidimensional  view  of  human  capa¬ 
bilities  provides  a  unique  opportunity  for  the  system 
engineer  to  identify  modifications  that  shift  operator 
workload  from  one  domain  to  another.  For  example,  technology 
designed  to  reduce  an  aviator's  need  to  maintain  physical 
control  of  system  functions  often  increases  the  aviator's 
role  as  a  monitor.  Thus,  advanced  technology  may  decrease 
operators'  psychomotor  workload  and  increase  their  cognitive 
workload.  Given  the  limited  capacity  of  human  cognitive 
ability,  system  designers  must  avoid  shifting  all  the  work¬ 
load  associated  with  aircraft  operations  into  the  cognitive 
domain  (or  any  other  single  domain,  for  that  matter) .  Thus, 
this  methodology,  with  its  second-by-second  estimate  of 
operator  workload,  will  allow  the  systems  engineer  to  better 
utilize  all  the  operators'  capabilities  and,  in  turn, 
increase  system  effectiveness. 


As  part  of  its  aviation  Product  Improvement  Program,  the 
Army  is  currently  considering  modifications  to  the  AH-64A 
helicopter.  The  Aviation  Systems  Command  (AVSCOM)  requested 
that  ARIARDA  adapt  the  LHX  workload  prediction  methodology  to 
the  AH- 64 A  system  to  assess  the  effect  that  the  proposed 
modifications  will  have  on  operator  workload.  In  response  to 
AVSCOM' s  request  for  support,  ARIARDA  tasked  Anacapa  to 
conduct  the  required  research. 


Research  Objectives 

The  primary  objective  of  the  AH-64A  workload  prediction 
research  is  to  determine  the  effect  that  advanced  technology 
is  likely  to  have  on  the  workload  of  AH-64A  crewmembers. 
Specifically,  the  research  is  designed  to: 

•  determine  the  operator  workload  for  the  current 
configuration  of  the  AH-64A  aircraft, 

•  identify  the  AH-64A  mission  functions  and  subsystems 
for  which  design  modifications  will  be  most  beneficial 
in  reducing  crew  workload,  and 

•  predict  the  effect  that  specific  design  modifications 
will  have  on  crew  workload. 

The  workload  predictions  yielded  by  the  methodology  will 
provide  an  objective  assessment  of  the  crewmember  workload  in 
the  current  configuration  of  the  AH-64A  aircraft  and  provide 
a  workload  criterion  for  evaluating  proposed  development  of 
the  helicopter.  This  input  should  be  valuable  in  improving 
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the  human-systems  interaction,  and  thus  improve  mission 
performance . 


Research  Approach 

The  research  for  meeting  these  objectives  has  been 
divided  into  three  projects: 

•  development  of  a  model  to  predict  AH-64A  crewmember 
workload, 

•  validation  of  the  AH-64  model,  and 

•  assessment  of  the  workload  effects  of  AH-64A 
modifications . 

In  previous  research,  a  comprehensive  mission/task/workload 
analysis  was  conducted  for  AH-64A  crewmembers.  The  results 
of  the  mission/task/workload  analysis  will  be  used  to  develop 
a  computer  model  for  predicting  baseline  AH-64A  crew  work¬ 
load.  The  workload  predictions  yielded  by  the  model  will 
then  be  validated.  Finally,  the  computer  model  will  be 
exercised  to  predict  the  effect  that  design  modifications  are 
likely  to  have  on  crew  workload  for  various  aircraft  configu¬ 
rations.  Each  of  the  projects  is  described  more  fully  below, 
following  a  brief  description  of  the  work  completed  under  the 
previous  contract . 


Background 

In  previous  research,  a  comprehensive  task/workload 
analysis  of  all  phases  of  the  AH-64A  attack  mission  was 
conducted.  A  composite  mission  scenario  was  developed  from 
five  mission  profiles  that  assumed  optimal  flight  conditions. 
In  the  composite  scenario,  the  pilot's  primary  function  was 
to  fly  the  aircraft  and  the  gunner's  primary  function  was  to 
acquire  and  engage  targets.  No  reconnaissance  or  team  leader 
functions  were  performed  by  the  crew.  During  the  analysis,  7 
mission  phases  were  identified  and  divided  into  52  unique 
mission  segments.  The  segments  were  further  divided  into  159 
unique  functions  with  688  individual  tasks  necessary  to  the 
mission.  The  subsystem,  crewmember,  and  time  for  each  task 
was  recorded.  The  results  of  the  AH-64A  analysis  were 
described  in  a  draft  technical  report  by  Szabo  and  Bierbaum 
(1986)  . 


Development  of  the  AH-64  Workload  Rredi.ct.iQn  Model 

The  development  of  a  computer  model  to  predict  workload 
for  the  crewmembers  in  the  current  configuration  of  the 


25 


AH-64A  is  the  first  project  in  this  research  area.  The 
principal  tasks  in  developing  the  model  are: 

•  establish  computer  files  for  the  mission/task/workload 
analysis  data, 

•  develop  function  and  segment  decision  rules, 

•  write  computer  programs  to  produce  estimates  of  work¬ 
load  for  the  AH-64A  baseline  configuration,  and 

•  review  the  model's  behavior  for  validity  and 
correctness . 

Each  of  these  tasks  is  described  briefly  below. 

The  initial  task  in  this  project  will  be  to  enter  the 
mission/task/workload  analysis  data  into  a  computer  data 
base.  This  computer  data  base,  when  combined  with  the 
function  and  segment  decision  rules,  will  form  the  input  to 
the  computer  model  for  AH-64A  workload  predictions. 

For  each  of  the  159  unique  functions,  a  function  summary 
sheet  will  be  developed  to  identify  the  specific  tasks  per¬ 
formed  by  each  crewmember.  Function  decision  rules  will  be 
written  to  identify  the  sequence  and  time  for  the  performance 
of  these  tasks.  Following  the  development  of  the  function 
summary  sheets  and  decision  rules,  segment  summary  sheets  and 
decision  rules  will  be  written.  The  segment  decision  rules 
will  specify  the  procedure  (sequence  and  time)  for  combining 
the  functions,  created  by  the  function  decision  rules,  to 
form  each  mission  segment. 

To  permit  an  automated  analysis  of  workload,  computer 
programs  will  be  developed  to  implement  the  decision  rules. 
The  mission/task  analysis  conducted  during  the  previous 
contract  used  a  top-down  approach  in  which  the  tasks  were 
identified  as  the  basic  elements  of  the  mission.  The 
computer  model  developed  during  this  project  will  use  a 
bottom-up  approach  in  which  the  tasks  (contained  in  the 
computer  files)  are  combined  to  form  functions  which,  in 
turn,  are  combined  to  form  segments.  The  time-based  function 
and  segment  decision  rules  are  the  blueprints  for  placing  the 
tasks  performed  by  each  crewmember  at  the  appropriate  point 
on  the  mission  timeline.  These  computer  programs  will 
simulate  the  sequence  of  tasks  that  each  crewmember  must 
perform  to  accomplish  the  mission.  From  this  simulation  the 
program  will  generate  total  workload  estimates  for  each  of 
the  five  workload  components  by  summing  the  individual 
workload  ratings  for  all  the  tasks  that  are  currently  being 
performed.  Total  component  workload  predictions  will  be 
generated  for  every  half-second  interval  in  the  segment. 
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After  the  computer  programs  are  written,  the  model  can 
be  exercised  to  produce  estimates  of  total  workload  asso¬ 
ciated  with  the  performance  of  concurrent,  as  well  as 
sequential  tasks  in  the  AH-64A  baseline  configuration.  The 
estimates  of  total  workload  for  each  workload  component 
(i.e.,  visual,  auditory,  kinesthetic,  cognitive,  and  psycho¬ 
motor)  will  identify  points  on  the  mission  timeline  at  which 
excessive  workload,  referred  to  as  "overload,"  will  occur. 
Four  different  indexes  of  overload  have  been  identified  and 
will  be  computed  by  the  program. 

After  the  computer  model  is  exercised,  any  mistakes  made 
in  creating  the  task/workload  analysis  data  base  will  be 
identified  and  corrected.  The  exercise  of  the  model  will 
produce  the  first  look  at  the  results  of  the  dynamic  combina¬ 
tion  of  tasks  and  functions  to  form  segments  in  the  model. 
This  computer  simulation  of  the  crew  members'  actions  during 
each  mission  segment  will  then  be  reviewed  by  subject  matter 
experts  to  ensure  that  the  model  conforms  with  typical 
crewmember  actions. 


Validation  of  the  AH-64A  Workload  Prediction  Model 

During  the  second  project,  the  AH-64A  workload  predic¬ 
tion  model  will  be  validated.  Validation  of  the  model  will 
consist  of  these  major  steps: 

•  evaluation  of  the  reliability  of  the  workload 
component  rating  scales  and  the  workload  predictors 
used  in  developing  the  model, 

•  validation  of  the  overall  workload  predictions  yielded 
by  the  model,  and 

•  refinement  of  the  model. 

Evaluation  of  the  use  of  the  workload  component  rating  scales 
will  be  accomplished  by  determining  (a)  the  interrater  reli¬ 
ability  for  the  rank  order  of  the  verbal  anchors  within  each 
7-point  scale,  and  (b)  the  interrater  reliability  for 
assigning  the  numerical  ratings  to  the  verbal  descriptors  of 
workload . 

Validation  of  the  overall  workload  predictions  yielded 
by  the  model  will  be  established  by  conducting  part-mission 
and  full-mission  simulation  research.  In  each  instance, 
predictions  of  workload  for  specific  tasks  will  be  compared 
with  objective  measures  of  primary  and  secondary  task  perfor¬ 
mance  and  subjective  measures  of  workload  such  as  the 
Subjective  Workload  Assessment  Technique  ratings  (Reid, 
Shingledecker,  &  Eggemeier,  1981)  or  the  NASA  Task  Load  Index 
ratings  (Hart  &  Staveland,  1987)  .  The  results  of  the 
validation  investigations  of  both  the  rating  scales  and  the 
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workload  predictions  will  subsequently  be  used  to  refine  the 
model . 


Workload  Analysis  in  Support  of  AH-64A  Development 

During  this  project,  the  AH-64A  computer  model  will  be 
exercised  to  predict  how  crew  workload  might  be  affected  by 
proposed  modifications  to  the  aircraft.  The  project  consists 
of  the  following  steps: 

•  identify  the  automation  options  proposed, 

•  conduct  a  task/workload  analysis  for  each  option, 

•  exercise  the  model  to  yield  revised  estimates  of 
workload,  and 

•  compare  the  estimates  of  workload  for  the  baseline  and 
automated  configurations. 

The  results  of  this  project  can  be  used  to  estimate  the 
effects  of  the  proposed  automation  options  on  the  workload  of 
the  crewmembers.  These  estimates,  in  turn,  will  assist 
design  engineers  in  determining  the  optimal  configuration  for 
the  aircraft . 


Results 


Work  Completed 

During  the  first  year  of  the  current  contract,  both  the 
mission/task/workload  analysis  and  the  decision  rules  were 
extensively  reviewed  and  revised.  In  addition,  a  preliminary 
version  of  the  computer  model,  using  a  Perkin-Elmer  mini¬ 
computer  and  FORTRAN  language,  was  developed.  The  model  was 
exercised  to  produce  preliminary  analyses  of  workload  for 
each  of  the  mission  segments. 

During  the  current  contract  year,  a  general  mission/ 
task/workload  analysis  software  system  was  developed  using  a 
Zenith  microcomputer  and  Turbo  Pascal  (see  Development  of  a 
Task  Analysis/Workload  (TAWL)  Software  System,  pp.  11-14  of 
this  report).  This  task  analysis/workload  software  system 
was  developed  and  refined  in  support  of  this  project  and 
other  projects  currently  being  conducted  by  Anacapa 
researchers . 

During  the  current  contract  year,  the  AH-64A  workload 
model  also  was  exercised  to  produce  mission  segment  print¬ 
outs.  These  printouts  were  reviewed  to  make  certain  that  the 
computer  model  accurately  simulated  the  function  and  segment 
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decision  rules.  In  some  cases,  the  function  and  segment 
decision  rules  were  revised. 


Work  Projected 

During  the  next  contract  year,  the  revised  model  will  be 
exercised  to  produce  estimates  for  each  of  the  52  unique 
mission  segments.  These  predictions  will  be  reviewed  to 
produce  a  final  baseline  version  of  the  model.  Results  will 
be  fully  described  in  a  research  report.  The  previously 
produced  draft  technical  report  (Szabo  &  Bierbaum,  1986)  will 
be  revised  to  reflect  the  final  version  of  the  model. 

A  research  plan  for  validating  the  model  will  be  devel¬ 
oped  and  submitted  to  ARIARDA  for  approval .  Upon  ARIARDA 
approval  of  the  research  plan,  the  validation  research  will 
begin.  As  the  refinement  of  the  model  continues,  the  model 
will  be  used  to  assess  the  effect  of  proposed  AH-64A  modifi¬ 
cations  on  operator  workload. 
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SURVEY  OF  HUMAN  FACTORS  MODELS  AND  METHODOLOGIES 
FOR  IMPROVING  MAINTAINABILITY  DESIGN 

Dr.  John  W.  Ruffner,  Project  Director 


Background 

Increasingly  complex  aviation  systems  are  being  devel¬ 
oped  to  enhance  the  ability  of  Army  aviators  to  fight  and 
survive  on  the  modern  battlefield.  However,  these  systems 
are  often  designed  with  little  regard  for  the  mental  and 
physical  capabilities  and  limitations  of  the  soldiers  who  are 
required  to  operate  and  maintain  them  (Neal,  Robinson, 

Takacs,  &  Rainwater,  1986) . 

Some  progress  has  been  made  in  designing  aviation 
systems  to  be  consistent  with  the  capabilities  and  limita¬ 
tions  of  the  operator;  however,  comparatively  little  atten¬ 
tion  has  been  paid  to  designing  systems  to  improve  their 
maintainability.  Traditionally,  the  maintainability  of  a 
system  is  given  the  lowest  priority  during  the  design 
process,  with  maintainability  being  secondary  to  performance, 
cost,  and  operability  criteria.  By  the  time  maintainability 
problems  are  identified,  changes  for  the  sake  of  efficient 
maintenance  are  often  not  feasible  (McDaniel  &  Askren,  1985)  . 
Furthermore,  efforts  to  increase  system  performance  and 
operability  often  result  in  added  system  complexity,  with  a 
concomitant  increase  in  maintenance  requirements. 

Maintenance  costs  are  often  the  most  important  element 
in  the  life-cycle  cost  estimates  for  an  aviation  system.  A 
typical  breakdown  is  approximately  15%  for  design,  35%  for 
production,  and  50%  for  operation  and  support  (Bond,  1987)  . 
The  proportion  of  the  Department  of  Defense  annual  budget 
required  for  maintenance  has  been  estimated  to  be  between  25% 
and  30%.  Furthermore,  the  total  maintenance  costs  of  a  piece 
of  equipment  throughout  its  life  cycle  are  often  expected  to 
exceed  its  acquisition  costs  (Christensen  &  Howard,  1981)  . 

•Smith,  Westland,  and  Crawford  (1970)  conducted  a  compre¬ 
hensive  review  of  the  maintainability  literature  available  at 
the  time.  They  noted  that  the  problem  of  maintaining  mili¬ 
tary  equipment  in  a  state  of  readiness  had  grown  to  enormous 
proportions  from  1950  to  1970,  and  that  the  dominant  cause  of 
the  maintenance  problem  was  the  progressive  and  rapid 
increase  in  equipment  complexity.  In  a  more  recent  review. 
Bond  (1987)  concluded  that  the  situation  had  not  improved 
since  that  time,  and  that  a  "persistent  maintenance  crisis" 
presently  exists  in  the  military. 
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This  situation  is  exacerbated  by  the  projected  shortfall 
in  the  number  of  military-aged  individuals  who  will  be  avail¬ 
able  for  maintaining  complex  aviation  systems.  The  personnel 
who  will  be  recruited  into  the  military  services  in  the  next 
20  years  will  be  fewer  in  number  and  of  lesser  capabilities, 
aptitudes,  and  competencies  than  at  present.  The  military 
services  will  have  to  compete  with  the  civilian  job  market 
for  the  most  capable  individuals  (Malone,  Heasly,  Waldeisen, 

&  Hayes,  1986) .  In  addition,  only  a  small  percentage  of 
military  technicians  serve  more  than  a  3-  or  4-year  enlist¬ 
ment  period.  This  makes  it  extremely  difficult  for  the 
majority  of  maintenance  technicians  to  achieve  the  skill 
level  required  to  maintain  complex  aviation  systems. 

Smith  et  al.  (1970)  identified  three  potential  solutions 
to  the  problem  of  adequately  maintaining  complex  military 
systems:  (a)  improve  technician  skills  through  training,  (b) 

improve  troubleshooting  aids,  and  (c)  improve  equipment 
design.  Prior  to  the  mid-1960s,  the  predominant  emphasis  was 
placed  on  improving  technician  skills,  with  secondary 
emphasis  on  performance  aids.  Little,  if  any,  attention  was 
paid  to  improving  equipment  design. 

As  Cunningham  and  Cox  (1972)  noted,  until  equipment 
complexity  became  overwhelming,  training  was  considered  the 
most  effective  method  of  minimizing  the  time  for  system 
maintenance.  In  1970,  Smith  et  al.  argued  that  efforts  to 
provide  better  training  were  not  succeeding  in  reducing  the 
maintenance  problem.  Inadequate  training  was  especially 
evident  for  troubleshooting  problems  that  account  for  more 
than  60%  of  corrective  maintenance  time  electronic 
equipment . 

To  improve  the  maintainability  of  equipment  while 
compensating  for  lower  technician  skill  levels,  maintenance 
technicians  were  supplied  with  improved  troubleshooting  aids 
(e.g.,  written  procedural  guides,  built-in  test  equipment, 
automatic  test  equipment) .  However,  periodic  surveys  have 
indicated  that  there  have  been  negligible  increases  in  the 
effectiveness  of  troubleshooting  manuals.  Even  the  most 
accurate  built-in  test  equipment  and  automatic  test  equipment 
miss  5  -  10%  of  system  faults,  thus  requiring  manual  trouble¬ 
shooting.  There  still  is  an  unacceptably  high  number  of 
false  alarms  and  unnecessary  replacement  of  functioning 
equipment,  resulting  in  time  wasted  in  unnecessary  mainte¬ 
nance  actions  and  increased  loads  on  the  rest  of  the 
logistics  system  (Maxion,  1984;  Coppola,  1984). 

Smith  et  al.  (1970),  Crawford  and  Altman  (1972),  and 
Potempa,  Lintz,  and  Luckew  (1975),  among  others,  strongly 
emphasized  the  need  to  improve  maintainability  by  influencing 
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the  design  of  systems  as  early  as  possible  during  the  acqui¬ 
sition  process.  They  stated  that  (a)  equipment  design  is  the 
most  important  factor  contributing  to  the  level  of  maintain¬ 
ability,  and  (b)  there  is  a  pressing  need  for  data,  methods, 
and  models  that  specify  human  factors  inputs  to  the  engineer 
during  system  development. 

More  recently,  the  Department  of  Defense  MIL-STD-470A 
(1983)  stated  that  manpower  and  personnel  shortages  are  of 
such  magnitude  that  the  maintainability  problem  must  be 
approached  through  the  design  process  as  well  as  through  the 
more  traditional  approaches  of  improving  training  and 
providing  job  performance  aids.  One  of  the  primary  objec¬ 
tives  of  the  Army's  Manpower  and  Personnel  Integration 
(MANPRINT)  program  is  to  influence  the  design  of  military 
systems  so  that  they  can  be  operated  and  maintained  in  the 
most  cost-effective  and  safest  manner  consistent  with  the 
manpower  structure,  personnel  aptitude  and  skill,  and 
training  resource  constraints  of  the  Army  (Department  of  the 
Army,  1987) . 


As  noted  above,  little  progress  has  been  made  in 
increasing  the  maintainability  of  systems  through  improve¬ 
ments  in  training  or  job  performance  (e.g.,  troubleshooting) 
aids.  Several  sets  of  maintainability  guidelines  have  been 
developed  and  published  in  the  last  30  years  (e.g.,  Folley  & 
Altman,  1956;  Department  of  Defense,  1984),  but  their  content 
cannot  be  applied  during  systematic  evaluations  of  various 
design  variables.  In  addition,  there  is  little  data  upon 
which  quantitative  maintainability  trade-offs  can  be 
calculated  between  equipment  designs. 

In  response  to  the  Army's  MANPRINT  initiative,  several 
models  and  procedural  methodologies  for  applying  knowledge 
about  the  capabilities  and  limitations  of  human  operators  and 
maintainers  to  the  design  of  military  systems  have  been 
developed  or  modified.  The  majority  of  this  work  has  been 
directed  toward  the  role  of  the  human  as  system  operator 
rather  than  as  system  maintainer.  There  is  a  mounting  body 
of  evidence  indicating  that  models  and  procedural  method¬ 
ologies  are  needed  that  can  be  applied  toward  improving  the 
maintainability  of  aviation  systems  as  early  as  possible  in 
the  system  design  and  acquisition  processes. 
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Project  Objective 


The  objective  of  this  project  is  to  identify  human 
factors  models  and  methodologies  that  may  be  used  to  improve 
the  maintainability  design  of  emerging  Army  aviation  systems. 


Project  Status 


Work  Completed 

Work  on  this  project  commenced  in  December,  1987.  The 
project  director  conducted  a  search  of  (a)  the  National 
Technical  Information  Service  and  the  Defense  Technical 
Information  Center  data  bases,  and  (b)  the  cumulative  indexes 
for  selected  scientific  publications.  As  a  result  of  this 
search,  approximately  100  documents  were  identified  that  are 
relevant  to  the  topics  of  aviation  maintenance  and  maintain¬ 
ability  design.  These  include  maintainability  engineering 
textbooks,  maintainability  guidelines,  military  standards  and 
handbooks,  Army  regulations,  field  manuals,  technical 
manuals,  literature  reviews,  technical  reports,  journal 
articles,  and  papers  presented  at  professional  meetings. 

Work  began  on  writing  a  draft  report  during  June,  1988. 
An  outline  of  the  report  was  reviewed  and  approved  by  the 
Army  Research  Institute  Aviation  Research  and  Development 
Activity  ( ARIARDA)  in  July,  1988.  The  report  is  organized 
into  four  sections.  The  first  section  describes  the  back¬ 
ground,  purpose  and  scope  of  the  survey.  The  second  section 
presents  a  brief  summary  of  Army  aviation  maintenance  levels 
and  tasks  and  an  overview  of  previous  maintainability  design 
research.  The  third  section  presents  a  detailed  summary  of 
human  factors  models  and  methodologies  that  may  be  used  to 
improve  the  maintainability  design  of  emerging  Army  aviation 
systems.  The  fourth  section  identifies  several  key  mainte¬ 
nance  and  maintainability  design  issues  in  Army  aviation  and 
presents  recommendations  for  research  to  address  these 
issues.  At  the  end  of  the  current  contract  year,  a  draft 
version  of  the  first  three  sections  of  the  report  was 
completed . 


Work  Projected 

A  draft  version  of  the  report  and  an  annotated  bibliog¬ 
raphy  summarizing  the  maintainability  design  literature 
reviewed  for  this  project  will  be  completed  and  submitted  for 
formal  ARIARDA  review  early  in  the  next  contract  year.  It  is 
anticipated  that  after  ARIARDA  reviews  the  report,  additional 
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research  in  the  areas  of  aviation  maintenance  and  maintain¬ 
ability  design  will  be  planned  and  conducted. 
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DESIGN  AND  APPLICATION  OF  FLIGHT  SYMBOLOGY 
Dr.  Richard  Weeter,  Project  Director 


Background 

The  AH-64A  attack  helicopter  is  the  first  Army  aircraft 
to  feature  the  Pilot  Night  Vision  System  (PNVS) .  The  PNVS  is 
a  display  system  designed  to  enable  crewmembers  to  conduct 
attack  missions  at  night  and  in  adverse  weather.  A  40°  hori¬ 
zontal  by  30°  vertical  field  of  view  is  presented  to  the 
pilot's  right  eye  by  a  1-inch  in  diameter  cathode  ray  tube 
mounted  on  the  pilot’s  helmet.  A  set  of  27  symbols  can  be 
projected  onto  this  field  of  view,  overlaid  on  external 
visual  imagery  produced  by  an  infrared  sensor.  The  symbology 
is  intended  to  provide  the  pilot  with  critical  flight  and 
targeting  information. 

The  PNVS  symbology  consists  of  27  alphanumeric  and 
shape-coded  symbols .  Many  of  the  symbols  are  adaptations 
from  traditional  electromechanical  flight  instruments  (e.g., 
the  heading  scale  and  lubber  line) .  Some  symbols  appear  at 
fixed  locations  on  the  displays.  Symbols  representing  air¬ 
craft  heading,  airspeed,  altitude,  engine  torque,  and  certain 
other  basic  flight  information  are  constantly  available  on 
the  display.  Other  symbols  are  dynamic  representations  of 
spatial  information  such  as  the  projected  center  line  of  the 
aircraft.  Such  dynamic  symbols  may  appear,  disappear,  and 
move  on  and  off  the  display  to  reflect  changes  in  aircraft  or 
sensor  orientation. 

The  pilot  can  select  four  subsets  or  operating  modes  of 
the  PNVS  symbology  to  reduce  obscuration  and  make  symbolic 
information  more  task-specific.  Each  mode  adds  symbology 
considered  critical  for  performing  a  particular  flight  or 
weapons  task  to  the  basic  flight  information  symbology. 

Selection  of  the  hover  mode  adds  a  velocity  vector  and 
an  acceleration  cue  to  aid  the  pilot  in  maintaining  a  hover. 
Selection  of  the  transition  mode  adds  a  horizon  line  to  the 
hover  mode  subset  and  is  used  when  changing  from  a  hover  to 
cruise  flight.  Once  cruise  flight  is  established,  selection 
of  the  cruise  mode  removes  the  velocity  vector  and  accelera¬ 
tion  cue  but  retains  the  horizon  line  in  addition  to  the 
basic  flight  symbology.  To  aid  the  pilot  in  maintaining  his 
position  or  in  returning  to  a  previous  location  and  heading, 
the  bob-up  mode  adds  the  velocity  vector,  acceleration  cue, 
command  heading,  and  hover  position  symbols  to  the  basic 
flight  information. 
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In  1977,  Schmitt  noted  that  little  research  had  been 
accomplished  that  might  contribute  to  an  evaluation  of 
symbology  formats  for  aircraft  visual  displays.  During  the 
development  of  the  AH-64A  PNVS  symbology,  Buckler  (1978a) 
described  the  state  of  research  comparing  different  symbology 
formats  as  "sorely  lacking."  Furthermore,  Buckler  (1978b) 
reported  that  reconr igurable  simulators  were  not  readily 
available  for  testing  alternative  symbol  designs.  As  a 
result,  the  PNVS  symbology  was  designed  largely  on  the  basis 
of  subject  matter  expert  (SME)  opinion.  Buckler  reported 
that  SMEs  (a)  analyzed  the  information  needed  by  the  crew  to 
perform  representative  AH-64A  flight  and  weapons  tasks,  and 
(b)  recommended  the  existing  symbology  format. 

To  date,  no  research  has  been  conducted  to  evaluate 
whether  the  AH-64A  PNVS  symbols  enhance  or  degrade  informa¬ 
tion  transfer  during  either  routine  or  critical  mission 
tasks.  Nevertheless,  the  Department  of  Defense  has  estab¬ 
lished  a  military  standard  for  symbology  formats  patterned 
after  the  current  AH-64A  PNVS  symbology  set  (Department  of 
Defense,  1984) .  Department  of  Defense  officials  intend  to 
revise  the  military  standard  as  data  become  available, 
implying  that  research  could  affect  the  design  of  future 
aircraft  display  symbology.  However,  Shrager  (1977) 
suggested  that  the  development  of  symbology  may  be  more 
"evolutionary"  than  systematic.  For  example,  symbology 
differing  from  the  AH-64A  PNVS  symbology  is  currently  being 
developed  for  the  Army's  MH-60K  and  MH-47E  special  operations 
helicopters  (International  Business  Machines,  1988),  but  no 
information  or  empirical  data  have  been  released  to  explain 
how  the  new  symbology  set  was  developed  or  how  it  will  affect 
crew  performance.  Schmitt  (1982)  cautioned  that  such  "ad  hoc 
integration"  of  display  information  creates  potential  flight 
safety  hazards. 


Problem 

Currently,  there  is  no  widely  accepted  research  method¬ 
ology  for  addressing  critical  symbology  design  issues. 

Hence,  there  are  no  empirically  valid  design  criteria  for  new 
aircraft  display  symbology.  Moreover,  there  is  no  widely 
accepted  research  methodology  for  evaluating  the  efficiency 
of  existing  symbology  sets.  The  first  problem  is  to  develop 
a  methodology  that  can  be  used  to  evaluate  the  efficiency  of 
the  AH-64A  PNVS  symbology  set. 

The  research  methodology  must  address  whether  the  symbol 
coding  dimensions  are  compatible  with  the  cognitive  processes 
of  AH-64A  crewmembers.  Ideally,  successive  experiments 
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employing  the  methodology  will  culminate  in  the  development 
of  symbol  and  display  format  design  criteria.  The  resultant 
symbology  should  (a)  be  compatible  with  known  capabilities 
and  limitations  of  aviators,  (b)  present  information  so  that 
it  can  be  interpreted  accurately  and  efficiently,  and  (c) 
complemrr4-  ’-ather  than  interfere  with  visual  information 
available  from  the  natural  external  visual  scene  and  from 
sensor-provided  imagery. 

The  Army  Research  Institute  Aviation  Research  and 
Development  Activity  (ARIARDA)  was  tasked  by  the  Army 
Aviation  Systems  Command  to  initiate  research  to  meet  these 
needs.  Anacapa  Sciences  researchers  began  work  on  the 
project  in  February,  1987. 


Project  Objectives 

The  three  objectives  of  this  project  are:  (a)  to 
develop  a  methodology  for  evaluating  aircraft  display  symbol¬ 
ogy,  (b)  to  conduct  empirical  evaluations  of  the  existing  AH- 
64A  PNVS  symbology,  and  (c)  to  identify  potential  design 
criteria  for  modifying  the  AH-64A  PNVS  symbology  and  for 
developing  future  aircraft  display  symbology. 


fiesestrch  .Approach 

Although  experimental  conditions  and  tasks  vary  widely, 
participants  in  selective  visual  attention  experiments  some¬ 
times  are  required  to  perform  visual  tasks  similar  to  those 
of  pilots  using  aircraft  visual  displays  (e.g.,  Lyon,  1987; 
Williams,  1982) .  Such  experiments  have  revealed  that  a 
number  of  factors  can  affect  attentional  performance  of 
visual  tasks.  For  example,  Eriksen  and  Hoffman  (1972) 
demonstrated  that  the  process  of  encoding  information  on 
visual  displays  is  affected  by  the  number,  nature,  and 
proximity  of  noise  elements.  Recently,  Lyon  (1987)  suggested 
that  the  rapid  attention  shift  component  of  selective  visual 
attention  may  be  a  measurable  aspect  of  skilled  performance 
in  vision-dependent  tasks.  These  findings  suggest  that 
selective  visual  attention  research  may  prove  useful  in 
evaluating  aircraft  visual  display  symbology. 

Currently,  a  research  plan  is  being  developed  for  con¬ 
ducting  selective  visual  attention  experiments  to  evaluate 
the  AH-64 A  PNVS  symbology.  The  proposed  research  will  inves¬ 
tigate  whether  the  selective  visual  attention  research 
methodology  can  contribute  to  the  development  of  evaluation 
and  design  guidelines  for  aircraft  display  symbology. 
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Project  Status 


Work  Completed 

At  the  close  of  the  current  contract  year,  project 
personnel  were  completing  revisions  to  two  documents.  The 
first  document  is  a  literature  review  of  factors  that  influ¬ 
ence  the  design  and  evaluation  of  aircraft  display  symbology. 
The  literature  review  emphasizes  research  that  evaluates  the 
effectiveness  of  symbology  coding  dimensions.  The  literature 
review  also  includes  a  bibliography  of  symbology  references 
and  a  list  of  possible  independent  variables  in  symbology 
experiments . 

The  second  document  is  a  research  plan  that  is  being 
developed  to  address  the  project  objectives  within  the 
constraints  of  available  Anacapa  and  ARIARDA  resources.  As 
noted  above,  the  theoretical  basis  of  the  research  plan  and 
the  paradigms  for  the  first  experiments  are  drawn  from  the 
selective  visual  attention  research  literature.  Two 
experiments  designed  to  evaluate  the  current  cueing  procedure 
for  the  AH-64A  PNVS  Cued  Line-of-Sight  (LOS)  symbol  are 
described  in  the  plan.  The  experiments  are  an  initial 
attempt  to  develop  an  empirical  method  of  evaluating  visual 
display  symbology. 

To  support  the  development  of  this  research  plan, 
project  personnel  duplicated  portions  of  videotape  recordings 
made  by  the  AH-64A  Video  Recorder  Subsystem  (VRS) .  The 
recordings  contain  sequences  of  PNVS  symbology  in  use  under 
operational  conditions.  In  addition,  ARIARDA  acquired  an 
experimental  workstation  to  support  this  research.  The 
workstation  consists  of  a  Mitsubishi  Diamond  Scan  monitor,  an 
80386  processor-based  Zenith  personal  computer,  and  addi¬ 
tional  computer  hardware  (one  360K  floppy  disc  drive,  one  64K 
RAM  cache  card,  one  4MB  internal  memory  card,  and  one  multi¬ 
function  card) .  The  experimental  hardware  has  been  installed 
and  tested.  Project  personnel  have  started  to  develop  the 
computer  programs  required  to  conduct  the  initial 
experiments . 


Work  Projected 

Project  personnel  will  complete  revisions  to  both 
project  documents.  Once  the  research  plan  has  been  approved 
by  ARIARDA,  the  experiments  will  be  conducted.  If  results 
from  the  first  two  experiments  indicate  that  the  selective 
visual  attention  research  methodology  is  successful,  Anacapa 
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researchers  will  design  a  series  of  selective  visual  atten¬ 
tion  experiments  to  evaluate  other  AH-64A  PNVS  symbols.  The 
experiments  will  increase  in  complexity,  progressing  from 
evaluations  of  individual  symbols  and  procedures  (e.g.,  the 
cueing  procedure  for  the  PNVS  Cued  LOS  symbol)  to  evaluations 
of  groups  of  symbols  superimposed  on  actual  PNVS  visual 
imagery  recorded  from  the  AH-64A  VRS. 
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NEW  FAST  DEVELOPMENT/VALIDATION 
Dr.  D.  Michael  McAnulty,  Project  Director 


Background 

The  Army's  original  selection  battery,  the  Flight  Apti¬ 
tude  Selection  Test  (FAST) ,  was  developed  in  response  to  the 
unacceptably  high  attrition  rates  in  the  flight  training  pro¬ 
gram  during  the  1950s.  The  FAST  comprised  two  overlapping 
batteries,  one  for  commissioned  officer  (CO)  applicants  and 
one  for  enlisted  and  civilian  applicants  to  the  Warrant 
Officer  Candidate  (WOC)  program.  Each  battery  yielded  a 
fixed  wing  and  a  rotary  wing  aptitude  score  for  each  appli¬ 
cant  (Kaplan,  1965) .  The  FAST,  implemented  in  1966,  resulted 
in  a  substantial  reduction  in  the  flight  training  attrition 
rates . 


In  1975,  the  U.S.  Army  Aviation  Center  requested  a 
revision  of  the  FAST  that  would  produce  a  single,  effective 
battery  with  fewer,  shorter,  and  more  reliably  scored  sub¬ 
tests  (Eastman  &  McMullen,  1978) .  Factor  analyses  and 
multiple  regression  analyses  were  used  to  select  7  of  the  12 
FAST  subtests  for  retention  in  the  revised  FAST  (RFAST) . 
Subsequently,  item  difficulties  and  item  discrimination 
coefficients  were  analyzed  to  select  the  specific  items  to  be 
retained  in  each  subtest.  The  RFAST,  implemented  in  1980,  is 
approximately  one-half  the  length  of  the  original  FAST. 

In  subsequent  research,  Lockwood  and  Shipley  (1984) 
found  that  6  of  the  7  subtests  had  adequate  internal  consis¬ 
tency  and  that  the  correlation  between  the  RFAST  score  and 
performance  in  initial  entry  rotary  wing  (IERW)  training  was 
statistically  significant.  They  concluded,  however,  that  the 
low  percentage  of  variance  accounted  for  by  the  RFAST  indi¬ 
cates  the  battery  has  limited  utility  in  predicting  IERW 
performance.  In  addition,  Smith  and  McAnulty  (1985)  found 
that  the  RFAST  has  marginal  retest  reliability  and  that  there 
was  a  large  increase  in  the  average  score  on  retesting,  indi¬ 
cating  a  need  for  an  equivalent  form  for  use  when  retesting 
is  required. 

Early  in  the  development  of  a  new  FAST  (NFAST)  battery, 
research  was  conducted  to  identify  the  ability  requirements 
for  the  successful  completion  of  IERW  training.  Experienced 
IERW  instructor  pilots  (IPs)  were  asked  (a)  to  identify  the 
tasks  that  are  most  indicative  of  successful  performance  in 
the  primary  and  instrument  phases  of  IERW,  and  (b)  to  judge 
the  type  and  importance  of  the  abilities  that  are  required  to 
perform  each  task.  The  task-ability  ratings  for  each  IP  were 
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then  transformed  to  a  normally  distributed,  equal-interval 
scale  using  the  method  of  successive  intervals  (McAnulty  & 
Jones,  1984).  Analyses  of  the  transformed  ratings  indicated 
that  24  abilities  in  the  psychomotor,  perceptual,  language, 
and  cognitive  domains  were  required  for  successful  perfor¬ 
mance  in  IERW .  These  analyses  were  used  to  design  a  test 
specifications  matrix  to  guide  the  development  of  the  NFAST 
(McAnulty,  Jones,  Cohen,  &  Lockwood,  1984). 


Need 

The  U.S.  Army  Research  Institute  Aviation  Research  and 
Development  Activity  (ARIARDA)  has  a  continuing  requirement 
to  evaluate  and  improve  the  tests  that  are  used  to  select 
applicants  for  the  Army  IERW  training  program.  As  indicated 
in  the  Background  section,  an  NFAST  battery  is  needed  to 
improve  the  reliability  and  validity  of  the  IERW  selection 
process  and  to  provide  an  equivalent  form  to  be  used  for 
retesting  IERW  applicants. 


Project  Objectives 

The  general  objective  of  this  project  is  to  develop, 
evaluate,  and  implement  a  more  effective  battery  of  IERW 
selection  tests.  The  specific  technical  objectives  of  the 
research  project  are  to: 

•  develop  experimental  tests  to  measure  the  abilities 
required  for  successful  IERW  performance, 

•  develop  two  alternate  forms  of  the  NFAST  battery, 

•  conduct  pre-operational  research  to  validate  and 
equate  the  alternate  forms  of  the  NFAST  battery, 

•  produce  and  implement  the  operational  versions  of  the 
NFAST  battery,  and 

•  evaluate  the  NFAST  battery  and  administrative  proce¬ 
dures  in  operational  use. 


Research  Approach 

This  project  is  a  continuation  of  the  ongoing  ARIARDA 
research  program  in  aviator  selection  and  classification 
(McAnulty,  1986) .  The  research  is  divided  into  three 
projects.  In  the  initial  project  (NFAST  Development),  an 
experimental  battery  of  aviation-related  ability  tests  will 
be  developed  and  administered  to  general  population  subjects. 
The  data  collected  during  the  experimental  administration 
will  be  analyzed  to  determine  the  psychometric  characteris¬ 
tics  of  the  new  tests  and  the  interrelationships  among  the 
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tests.  The  results  of  these  analyses  will  be  used  to  develop 
two  alternate  forms  of  an  NFAST  validation  battery. 

The  second  research  project  (NFAST  Validation)  will  be  a 
predictive  validity  investigation.  The  pre-operat ional 
validation  research  will  be  conducted  to  (a)  determine  the 
relationship  between  the  NFAST  tests,  other  predictor  data, 
and  performance  in  IERW  training,  and  (b)  equate  the  alternate 
forms  of  the  battery  on  a  large  sample  drawn  from  the  target 
population  of  flight  students.  During  this  project,  measures 
of  IERW  performance  will  be  identified  and  evaluated  as  flight 
training  criteria.  The  results  of  the  validation  analyses 
will  be  used  to  produce  two  alternate  forms  of  an  operational 
NFAST  battery. 

The  third  research  project  (NFAST  Implementation)  will 
be  the  implementation  of  the  NFAST.  All  ancillary  materials 
(machine  scorable  answer  sheets,  administrative  manuals, 
scoring  and  equating  manuals,  selection  criteria)  will  be 
developed  and  delivered  for  operational  use.  After  the  NFAST 
is  implemented,  a  sample  of  field  data  will  be  analyzed  to 
ensure  that  the  psychometric  characteristics  of  the  opera¬ 
tional  battery  are  not  significantly  different  from  the 
validation  battery.  Finally,  data  will  be  collected  and 
analyzed  to  ensure  that  the  tests  are  being  administered 
properly  and  that  the  test  scores  are  being  used  appropri¬ 
ately  in  the  selection  process. 


Project  Stat.us 

Work, ...Completed: _ NFAST  Development 

The  NFAST  Development  project  has  been  completed.  Nine 
new  tests  were  developed  for  the  experimental  NFAST  battery. 
Eight  tests  were  each  designed  to  measure  a  unique  ability 
and  one  test  was  designed  to  measure  a  complex  of  abilities 
required  for  the  successful  completion  of  IERW  training. 

Four  standardized  tests  were  also  included  in  the  battery  as 
marker  variables.  The  7-hour  experimental  battery  was 
administered  to  290  general  population  subjects  at  three 
military  installations  in  the  southeastern  United  States. 

The  results  indicate  that  the  complex  ability  test  and 
six  of  the  unique  ability  tests  assess  reliable  individual 
differences  in  the  abilities  of  interest.  The  average 
difficulty  levels  of  the  seven  tests  are  near  the  optimum 
level  of  .50;  the  test  variances  indicate  the  measurement  of 
substantial  individual  differences;  and  the  estimates  of 
reliability  are  acceptable  when  test  length  and  the  design 
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specifications  are  considered.  The  remaining  two  unique 
ability  tests  had  undesirable  psychometric  characteristics  or 
did  not  contribute  any  unique  variance  to  the  factor  struc¬ 
ture  of  the  battery.  A  technical  report  (McAnulty,  Cross,  & 
Jones,  1986)  that  describes  the  results  of  this  research  was 
prepared  and  submitted  to  ARIARDA  for  review. 

The  research  results  were  used  to  develop  two  alternate 
forms  of  the  NFAST  validation  battery.  Each  form  consists  of 
modified  versions  of  the  complex  test  and  the  six  unique 
ability  tests  that  had  acceptable  psychometric  characteris¬ 
tics.  In  general,  the  validation  battery  tests  are  approxi¬ 
mately  two-thirds  the  length  of  the  experimental  battery 
tests.  The  alternate  forms  of  five  of  the  tests  have  approx¬ 
imately  50%  of  the  items  in  common.  The  complex  ability  test 
forms  and  one  of  the  unique  ability  test  forms  do  not  have 
any  identical  items.  Finally,  a  knowledge  test  of  helicopter 
operations  and  aerodynamic  principles  was  adapted  from  the 
RFAST  battery  for  inclusion  in  the  validation  battery.  The 
items  on  the  knowledge  test  are  identical  on  both  forms. 

Each  form  of  the  validation  battery  requires  approximately 
four  hours  to  administer. 


Mor,k.-,Comple,.t..e.d,; _ NFAST  Validation 

The  NFAST  Validation  project  activities  are  partially 
completed.  Between  March  and  October  1987,  the  alternate 
forms  of  the  NFAST  battery  were  administered  to  approximately 
95%  of  the  CO  and  WOC  flight  students  during  the  first  two 
weeks  of  IERW  training.  When  the  test  administration  phase 
was  terminated,  complete  and  usable  test  data  were  collected 
from  377  CO  and  341  WOC  students;  362  and  356  students  took 
Form  E  and  Form  F  of  the  battery,  respectively. 

Analyses  of  the  test  data  indicate  that  the  target  popu¬ 
lation  performance  on  the  validation  battery,  excluding  the 
helicopter  knowledge  test,  is  similar  to  the  general  popula¬ 
tion  performance  on  the  experimental  battery.  The  average 
difficulty  levels  are  near  .50  despite  the  more  restrictive 
time  limits  that  were  imposed  on  the  validation  tests,  and 
the  variances  indicate  that  substantial  individual  differ¬ 
ences  in  ability  are  being  measured  by  the  tests.  The 
internal  consistency  estimates  of  reliability  are  also  within 
an  acceptable  range.  Performance  on  the  two  forms  of  the 
battery  is  very  similar  except  for  one  of  the  unique  ability 
tests.  Test  performance  by  the  CO  and  WOC  students  is  also 
quite  similar,  although  the  CO  students  scored  signif icantly 
higher  (p  <  .01)  on  four  of  the  tests. 
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The  test  results  on  the  helicopter  knowledge  test 
adapted  from  the  RFAST  indicate  that  the  test  is  not  diffi¬ 
cult  and  that  there  is  limited  variability  in  the  scores. 

WOC  students  scored  significantly  higher  on  the  test  than  the 
CO  students.  However,  there  is  no  difference  in  performance 
by  either  student  group  on  the  two  forms  of  the  test .  Since 
the  test  items  are  identical  on  both  forms,  this  result 
indicates  there  was  no  systematic  sampling  bias  in  terms  of 
aviation-related  knowledge  in  the  assignment  of  students  to 
the  alternate  forms  of  the  NFAST  battery. 

The  collection  of  IERW  performance  measures  is  nearly 
completed.  Academic  test  grades  and  end-of-IERW-phase  flight 
checkrides  are  being  collected  from  the  Academic  Records 
Division  as  the  students  complete  training.  Data  are  also 
being  collected  on  the  number  of  flight  hours  each  student 
required  to  complete  each  phase  of  training.  Finally, 
changes  to  the  class  rosters  are  being  monitored  to  identify 
tested  students  who  have  been  eliminated  from  training  or  who 
have  moved  to  a  different  class.  The  reasons  for  these 
administrative  changes  (e.g.,  some  setbacks  may  be  due  to 
student  flight  deficiencies  and  others  may  be  due  to  a  short¬ 
age  of  IPs)  are  also  being  recorded. 


W.o.rk.-Exc),j.g.c.L,e.cii _ NFAS.I.. .  V.a  1  i  da  tl.Q.n 

IERW  performance  data  will  be  collected  until  all  the 
students  in  the  test  data  base  have  either  graduated  or 
attrited  from  flight  training  (approximately  one  year  after 
the  last  battery  administration) .  When  the  performance  data 
base  has  been  established,  appropriate  statistical  analyses 
will  be  conducted  to  determine  the  psychometric  characteris¬ 
tics  of  the  performance  measures,  the  interrelationships 
among  the  performance  measures,  and  the  correlations  between 
the  predictor  and  performance  measures .  Preliminary  analyses 
(a  =  549)  indicate  that  a  subset  of  tests  from  the  NFAST 
battery  will  significantly  improve  the  effectiveness  of  the 
selection  procedures  used  for  the  IERW  course.  The  results 
of  the  final  analyses  will  be  used  to  produce  two  alternate 
forms  of  the  NFAST  for  operational  use.  The  development  of 
the  operational  batteries  should  be  completed  oy  April  1989. 


Work  Projected;  NFAST  Implementation 

The  NFAST  Implementation  project  has  not  been  formally 
initiated,  although  a  prototype  administration  manual  was 
developed  for  use  with  the  validation  battery.  The 
administration  manual  will  be  modified  for  use  with  the 
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operational  battery.  Additional  ancillary  materials,  such  as 
a  scoring  and  equating  manual  and  selection  criteria 
guidelines,  will  be  developed.  The  operational  batteries  and 
ancillary  materials  will  be  delivered  to  the  U.S.  Army 
Soldier  Support  Center  for  implementation. 

After  the  NFAST  is  in  operational  use,  follow-on 
research  is  planned  to  ensure  that  applicant  performance  on 
the  test  batteries  is  within  acceptable  limits,  that  adminis¬ 
trative  procedures  are  being  followed,  and  that  the  selection 
criteria  are  valid.  Depending  on  the  results  of  the  pre¬ 
ceding  research,  it  may  be  necessary  to  conduct  a  second 
validation  effort  using  an  unrestricted  sample  (i.e.,  not 
already  selected  for  flight  training)  of  IERW  applicants. 
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DEVELOPMENT  OF  A  PEER  COMPARISON  PROGRAM 
Dr.  D.  Michael  McAnulty,  Project  Director 


Background 

This  project  was  initiated  in  response  to  a  request  from 
the  School  Secretary,  U.S.  Army  Aviation  Center,  Fort  Rucker, 
Alabama,  for  support  in  developing  an  algorithm  to  select 
course  honor  graduates  based  on  the  "whole  person"  concept. 
The  School  Secretary  wanted  to  augment  the  academic  grade 
criterion  used  to  select  honor  graduates  in  the  Aviation 
Officer  Advanced  Course  (AVNOAC) ,  a  five-month  officer 
training  course  for  captains  and  promotable  first  lieu¬ 
tenants.  The  purposes  of  the  augmented  program  are: 

•  to  motivate  students  to  maximize  their  military  and 
academic  efforts  during  the  course,  and 

•  to  identify  students  who  have  high  potential  as  Army 
aviation  officers  at  an  early  stage  of  their  careers. 

Specifically,  the  School  Secretary  was  interested  in 
using  peer  assessments  by  the  AVNOAC  students  as  a  component 
in  the  honor  graduate  selection  algorithm.  The  peer  assess¬ 
ments  were  to  evaluate  aspects  of  the  students'  performance 
that  were  not  reflected  in  their  academic  scores.  Instructor 
ratings  were  not  considered  as  a  criterion  component  because 
of  the  limited  interaction  between  the  school  cadre  and  the 
students . 


Project  Objectives 

Following  a  review  of  the  peer  assessment  literature  and 
the  AVNOAC  syllabus,  a  peer  comparison  (PC)  methodology  was 
proposed  for  use  in  the  AVNOAC.  The  School  Secretary  agreed 
to  support  the  following  research  objectives: 

•  identify  the  most  important  military  qualities  that 
can  be  assessed  by  peers  during  the  AVNOAC, 

•  develop  the  PC  instruments  and  procedures  for  use  in 
the  AVNOAC,  and 

•  experimentally  evaluate  the  PC  technique  prior  to 
implementation . 


Research  Approach 

The  research  approach  was  divided  into  three  phases  that 
correspond  to  the  research  objectives.  During  Phase  1,  a 


military  qualities  survey  was  administered  to  identify  the  most 
important  qualities  that  can  be  assessed  by  peers  during  the 
AVNOAC .  The  survey  asks  senior  aviation  officers  to  rate  a 
list  of  primary  military  qualities  as  potential  dimensions  for 
evaluating  student  performance  and  for  identifying  students 
with  high  career  potential.  The  survey  data  provide  the  infor¬ 
mation  needed  to  develop  the  PC  instruments. 

During  Phase  2,  three  project  assessment  instruments 
were  developed:  the  PC  form  to  be  completed  by  the  class 
members  to  evaluate  their  peers,  a  faculty  advisor  rating 
(FAR)  form  to  be  completed  by  each  class  member's  training 
officer,  and  a  student  critique  to  be  completed  by  the  stu¬ 
dents  to  evaluate  the  PC  instruments  and  procedures.  During 
Phase  3,  the  PC  technique  was  administered  and  evaluated. 


Project  Status 


Work  Completed 

Phase  1:  Military  qualities  survey.  Following  a  search 
of  the  literature  and  a  review  of  current  Army  student  eval¬ 
uation  dimensions,  definitions  of  14  primary  military  quali¬ 
ties  (e.g.,  adaptability,  initiative,  judgment,  leadership, 
and  responsibility)  were  compiled  for  evaluation  by  senior 
aviation  officers.  Several  important  military  qualities  were 
excluded  from  the  survey  because  they  are  evaluated  by 
academic  scores  or  are  unlikely  to  be  demonstrated  during  the 
AVNOAC.  Sixteen  senior  Army  aviation  officers  were  asked  to 
rate  each  quality  on  the  following  four  scales: 

•  importance  to  the  performance  of  captains, 

•  importance  to  the  performance  of  senior  officers, 

•  probability  of  demonstration  during  the  AVNOAC,  and 

•  degree  of  overlap  with  the  other  qualities. 

Eleven  surveys  were  completed  and  returned.  Three  of 
the  qualities  (leadership,  judgment,  and  responsibility )  had 
consistently  high  ratings  and  were  selected  as  PC  dimensions. 
Seven  of  the  qualities  were  clearly  perceived  as  being 
inappropriate  PC  dimensions.  Appearance  and  cooperation  were 
selected  from  the  remaining  four  qualities  as  two  additional 
PC  dimensions. 

Phase  2:  Form  development.  Three  research  forms  were 
developed  for  use  in  this  project.  The  PC  form  was  developed 
from  (a)  the  results  of  the  military  qualities  survey,  (b)  a 
combination  of  the  peer  nomination  and  peer  ranking  tech¬ 
niques  (e.g.,  Kane  &  Lawler,  1978),  and  (c)  the  psycho¬ 
physical  method  of  paired  comparisons  (Engen,  1971).  On  the 
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PC  form,  each  section  member  (a  class  is  divided  into  two 
sections)  is  required  to  nominate  and  rank  order  five  peers 
on  the  basis  of  their  potential  as  Army  aviation  officers. 

The  section  member  then  makes  paired  comparisons  of  the 
nominees  on  the  five  military  qualities  that  were  selected 
from  the  military  qualities  survey. 

PC  scores  are  computed  for  each  peer  by  first  summing 
the  rank  score  (five  points  for  first  rank,  four  points  for 
second  rank,  ...,  one  point  for  fifth  rank)  from  each 
nominating  section  member.  The  summed  rank  scores  are  then 
added  to  the  number  of  favorable  comparisons  the  peer 
received  on  each  military  quality.  Finally,  the  total  is 
divided  by  the  maximum  possible  score  to  enable  direct 
comparisons  between  sections  with  unequal  numbers  of 
students.  The  PC  scores  can  range  from  0.0  (no  nominations) 
to  1.0  (ranked  first  by  all  section  members  and  always 
favorably  compared  with  the  other  nominees) .  Because  each 
section  member  nominates  five  peers  out  of  approximately  50 
students,  a  PC  score  greater  than  .20  probably  represents  a 
consensus  among  the  section  members  that  the  student  has  high 
potential  as  an  Army  aviation  officer. 

A  FAR  form  was  developed  to  obtain  independent  evalua¬ 
tions  of  the  students'  potential  as  Army  aviation  officers. 
Each  AVNOAC  faculty  advisor  usually  supervises  six  or  seven 
students.  On  the  FAR,  the  advisors  estimate  the  officer 
potential  of  their  students  by  assigning  them  percentile 
ranks  in  an  average  group  of  100  captains. 

Finally,  a  student  critique  form  was  developed  to  ascer¬ 
tain  student  attitudes  toward  the  peer  comparison  program. 

The  students  are  asked  to  rate  the  fairness,  utility,  aver¬ 
siveness,  and  difficulty  of  various  aspects  of  the  program. 
They  are  also  asked  to  express  their  opinions  about  the 
implementation  of  the  program  and  to  offer  recommendations 
for  improving  the  program. 

Phase  3:  Experimental  administration-1.  Peer  compar¬ 
isons  were  collected  on  an  experimental  basis  (i.e.,  the  PC 
scores  were  not  used  to  select  honor  graduates)  from  Sections 
1  (H  =  38)  and  2  (n  =  40)  of  AVNOAC  85-2  in  July  1985.  A 
second  set  of  PC  ratings  and  the  student  critiques  were 
collected  from  Section  1  (n  =  33)  and  Section  2  (n  =  28) 
approximately  one  month  later.  The  faculty  advisors 
completed  the  FARs  immediately  after  graduation.  In 
addition,  the  final  academic  averages  (AVGs)  were  obtained 
from  the  School  Secretary's  off.'  e. 
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The  scores  for  the  first  and  second  data  collections 
were  highly  correlated  (Section  1  =  .96  and  Section  2  =  .86), 
indicating  the  stability  of  the  ratings  over  time.  Because 
of  the  high  correlations,  the  ratings  from  the  two  data 
collections  were  combined  into  a  single  PC  score  for  each 
peer.  The  PC  scores  ranged  from  .00  to  .48  in  Section  1  and 
from  .00  to  .36  in  Section  2.  Four  peers  in  Section  1  and 
three  peers  in  Section  2  received  PC  scores  greater  than  .20. 
A  majority  of  the  PC  scores  in  both  sections  were  between  .00 
and  .05.  The  scores  indicate  a  high  consensus  among  the 
members  of  the  class  in  identifying  peers  with  the  highest 
potential  as  aviation  officers. 

The  PC  scores  were  then  correlated  with  the  FARs  and 
AVGs .  For  Sections  1  and  2,  respectively,  the  PC  correla¬ 
tions  are  .45  and  .33  with  the  FAR,  and  .55  and  .30  with  the 
AVG .  These  correlations  are  sufficiently  high  to  show  an 
expected  relationship  between  observations  of  the  same  indi¬ 
viduals.  At  the  same  time,  the  correlations  are  sufficiently 
low  to  indicate  that  the  PC  score  is  measuring  a  unique 
perspective  of  the  class  members.  The  correlations  between 
the  FAR  and  AVG  are  .76  and  .59  in  Sections  1  and  2,  respec¬ 
tively.  This  result  probably  indicates  that  the  faculty 
advisors  were  depending  upon  the  academic  average  as  a 
primary  source  of  information  in  making  their  ratings. 

Finally,  the  responses  to  the  PC  critique  were  tabu¬ 
lated.  The  overall  reaction  of  the  class  members  to  the  PC 
program  was  negative:  a  majority  indicated  that  the  PC  was 
very  biased,  slightly  or  not  at  all  useful,  and  slightly  or 
not  at  all  predictive  of  future  performance.  Furthermore, 

72%  of  the  respondents  were  either  very  or  extremely  unfavor¬ 
able  toward  any  potential  implementation  of  the  program.  The 
responses  to  the  other  critique  items  reflected  combinations 
of  positive,  negative,  and  neutral  attitudes,  without  any 
attitude  representing  a  majority  opinion. 

The  results  of  the  first  administration  indicated  that 
the  PC  technique  was  a  potentially  useful  procedure  for 
identifying  the  class  members  with  the  highest  potential  as 
Army  aviation  officers,  although  the  students  were  critical 
of  its  use.  There  were,  however,  several  problems  with  the 
first  administration.  First,  the  students  were  not  advised 
about  the  PC  program  before  the  experimental  administration. 
Second,  a  concurrent  but  surreptitious  attempt  by  the  class 
leaders  to  evaluate  the  section  members  was  discovered  just 
before  the  second  data  collection.  Both  of  these  problems 
may  have  affected  the  students'  attitudes  about  class  evalua¬ 
tions.  Finally,  the  period  of  time  that  elapsed  between  the 
first  and  second  PC  administrations  was  too  short  to  evaluate 
the  stability  of  the  peer  assessments. 
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Phase  3:  Experimental  administration  2.  The  second 
experimental  administration  was  designed  as  a  replication  of 
the  first  administration,  with  the  following  changes: 

•  students  were  advised  in  advance  of  the  research, 

•  other  non-academic  evaluations  were  prohibited, 

•  three  months  elapsed  between  the  initial  and  final 
data  collection, 

•  the  military  quality  definitions  were  modified 
slightly, 

•  the  order  of  presentation  of  the  military  qualities 
and  nominee  pairs  was  completely  counterbalanced,  and 

•  a  new  roster  coding  system  was  instituted  to  protect 
student  privacy. 

In  December  1985,  usable  PC  ratings  were  collected  from 
48  students  in  Section  1  and  48  students  in  Section  2  of 
AVNOAC  86-1.  In  March  1986,  47  students  in  Section  1  and  44 
students  in  Section  2  completed  usable  PC  ratings  and  student 
critiques.  FARs  were  completed  by  most  of  the  faculty 
advisors  shortly  after  graduation  and  the  AVGs  were  collected 
from  the  School  Secretary's  office. 

Two  types  of  reliability  coefficients  were  computed  on 
the  AVNOAC  86-1  ratings.  First,  the  correlations  between  the 
initial  and  final  ratings  are  .79  in  Section  1  and  .93  in 
Section  2,  indicating  the  stability  of  the  ratings  across  a 
period  of  approximately  three  months.  Second,  split-half 
(odd-even)  coro.elat ions  for  each  data  collection  for  each 
section  were  computed  to  evaluate  the  internal  consistency  of 
the  ratings.  The  correlations  are  .71  and  .76  for  Section  1, 
and  .93  for  both  data  collections  for  Section  2.  The  relia¬ 
bility  coefficients  are  acceptable  in  all  cases,  although 
they  are  obviously  higher  in  Section  2.  Because  of  the  high 
correlations,  the  ratings  from  the  two  data  collections  were 
combined  into  a  single  PC  score  for  each  peer  in  each 
section . 

The  PC  scores  ranged  from  .00  to  .24  in  Section  1  and 
from  .00  to  .47  in  Section  2.  Four  peers  in  Section  1  and 
two  peers  in  Section  2  received  PC  scores  greater  than  .20. 

As  in  class  85-2,  a  majority  of  the  PC  scores  in  both  sec¬ 
tions  were  between  .00  and  .05.  The  scores  indicate  a  high 
consensus  among  the  members  of  Section  2  in  identifying  the 
two  peers  (PCs  =  .42  and  .47)  with  the  highest  potential  as 
aviation  officers.  The  PC  scores  in  Section  1  also  distin¬ 
guish  the  peers  having  the  highest  potential,  even  though  the 
PC  scores  are  much  lower.  The  lower  scores  could  be  an  arti¬ 
fact  of  the  methodology  if  there  are  more  than  five  peers 
with  high  potential  who  are  not  substantially  different  from 
each  other. 


55 


The  PC  scores  were  then  correlated  with  the  FARs  and 
AVGs.  For  Sections  1  and  2,  respectively,  the  PC  correla¬ 
tions  are  .02  and  .30  with  the  FAR,  and  .24  and  .27  with  the 
AVG .  These  correlations  indicate  that  the  PC  score  is 
measuring  a  different  aspect  of  the  class  members'  perfor¬ 
mance  during  the  AVNOAC .  The  .02  correlation  between  the  FAR 
and  PC  in  Section  1  is  partially  attributable  to  the  highly 
skewed  distribution  of  FARs.  The  correlations  between  the 
FAR  and  AVG  are  .53  in  both  sections.  The  FAR-AVG  correla¬ 
tions  probably  indicate  that  the  faculty  advisors  used  the 
academic  average  as  a  source  of  rating  information. 

Finally,  the  PC  critique  responses  from  class  86-1  were 
negative  overall,  but  not  as  negative  as  class  85-2.  A 
majority  of  the  86-1  respondents  indicated  that  the  PC  was 
either  slightly  or  not  at  all  useful  for  selecting  AVNOAC 
honor  graduates;  the  respondents  were  approximately  evenly 
divided  on  the  issues  of  PC  fairness,  bias,  and  predict¬ 
ability  of  future  Army  performance.  Ratings  of  the  adequacy 
of  definitions  and  the  difficulty  of  nominating,  ranking,  and 
comparing  peers  were  very  similar  to  the  85-2  results. 

Despite  the  slight  positive  shift  in  attitude  toward  the  PC 
program,  69%  of  the  respondents  were  still  either  very  or 
extremely  unfavorable  toward  any  potential  implementation  of 
the  program. 

The  results  of  the  86-1  data  collection  support  the 
conclusion  drawn  from  the  85-2  results:  the  PC  technique  is 
a  potentially  useful  procedure  for  identifying  the  peers  with 
the  highest  potential  as  Army  aviation  officers,  at  least  in 
terms  of  the  reliability  of  the  ratings.  There  was  a  con¬ 
sensus  identification  of  the  peers  with  the  highest  potential 
and  the  ratings  were  generally  consistent  over  a  3-month  data 
collection  interval.  Longitudinal  research  is  required  to 
determine  the  validity  of  the  PC  technique  for  predicting 
future  performance.  Similar  to  the  85-2  students,  the  86-1 
students  found  the  rating  procedure  to  be  aversive  and  were 
unfavorable  toward  the  implementation  of  the  PC  technique. 


Work  Projected 

Only  one  activity  remains  to  be  completed  on  the  PC 
project.  A  report  will  be  prepared  to  present  the  results  of 
the  research  to  the  School  Secretary’s  office.  The  report 
will  include  recommendations  regarding  the  implementation  of 
the  PC  program. 
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EVALUATION  OF  THE  AVIATION  RESOURCE  MANAGEMENT  SURVEY 

(ARMS)  CHECKLIST 

Dr.  John  W.  Ruffner,  Project  Director 


Background 

According  to  the  Army’s  "total  force"  concept.  Reserve 
Component  (RC)  aviators  serving  in  the  U.S.  Army  Reserve 
(USAR)  and  the  Army  National  Guard  (ARNG)  are  required  to 
train  to  the  same  standards  and  to  maintain  the  same  level  of 
flight  proficiency  and  flight  safety  as  aviators  serving  in 
the  Active  Component.  RC  aviators  must  meet  these  require¬ 
ments  with  limited  resources.  Therefore,  the  individuals  who 
are  responsible  for  planning,  implementing,  and  evaluating  RC 
training  must  manage  the  available  resources  (e.g.,  aircraft, 
training  time,  flying  hours,  instructor  pilots)  efficiently. 

One  of  the  ways  that  the  Army  helps  RC  training  managers 
achieve  efficiency  is  through  evaluation  visits  from  Aviation 
Resource  Management  Survey  (ARMS)  teams.  U.S.  Army  Forces 
Command  (FORSCOM)  Regulation  350-3  (1984)  states  that  the 
general  purposes  of  the  ARMS  are  to  evaluate  the  management 
of  unit  aviation  programs,  to  identify  management  practices 
that  require  improvement,  and  to  provide  staff  assistance  as 
necessary.  As  defined  by  FORSCOM,  the  ARMS  has  four  specific 
objectives : 

•  to  help  commanders  identify  strengths  and  weaknesses 
in  all  aviation-related  programs; 

•  to  assess  an  aviation  support  facility's  capacity  to 
support  the  training  of  units  assigned  to  the 
facility; 

•  to  assess  the  aviation  unit's  capabilities  (a)  to 
operate  safely,  efficiently,  and  effectively,  and  (b) 
to  maintain  aviation  resources  apart  from  the  support 
facility  while  accomplishing  its  mobilization  mission; 
and 

•  to  identify  problems  and  coordinate  assistance  re¬ 
quired  to  solve  problems  that  are  beyond  the  facility 
commander's  or  unit  commander's  sphere  of  authority. 

The  Deputy  Chief  of  Staff  for  Training  (DCST)  in  each  of 
the  five  Continental  U.S.  Armies  (CONUSAs)  is  responsible  for 
conducting  ARMS  evaluations.  According  to  FORSCOM  Regulation 
350-3  (1984),  an  ARMS  is  to  be  conducted  at  least  once  a  year 
for  each  USAR  facility  and  unit,  and  at  least  once  every  two 
years  for  each  ARNG  facility  and  unit  within  the  CONUSA. 
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Problem 


Each  CONUSA  has  its  own  procedure  for  carrying  out  the 
ARMS  evaluation  program.  There  is  a  lack  of  standardization 
across  the  CONUSAs  in  (a)  the  functional  areas  (e.g.,  safety, 
standardization,  and  training)  that  are  evaluated;  (b)  the 
procedures  used  by  the  ARMS  teams  to  assess  the  facilities 
and  units;  and  (c)  the  standards  for  acceptable  performance. 

The  first  U.S.  Army  DCST,  Aviation  Division,  has 
developed  a  checklist  to  be  used  by  the  ARMS  team  during  its 
evaluation  visits.  The  checklist  originally  was  published  in 
October  1983,  and  republished  in  August  1985,  as  First  Army 
Pamphlet  95-1.  The  checklist  draws  from  two  sources:  (a) 
FORSCOM  Form  14-1-R  "Reserve  Component  Aviation  Resource 
Management  Checklist"  (1980),  and  (b)  the  U.S.  Army  Safety 
Center  "Guide  to  Aviation  Resources  Management  for  Aircraft 
Mishap  Prevention"  (1984). 

The  First  Army  checklist  contains  670  items  divided  into 
the  following  11  major  functional  areas  of  evaluation: 

•  Aviation  Safety  Management, 

•  Facility/Unit  Operations, 

•  Aviation  Standardization  and  Training, 

•  Aircraft/Flightline  Operations, 

•  Aeromedical  Management, 

•  Aircraft  Crash  Rescue  and  Fire  Fighting, 

•  Petroleum,  Oil,  and  Lubricants, 

•  Maintenance  Management, 

•  Aviation  Armament, 

•  Aviation  Life  Support  Equipment,  and 

•  Physical  Security. 

Each  checklist  item  describes  a  specific  deficiency  that  may 
result  in  (a)  the  failure  of  a  facility  to  accomplish  its 
mission  of  supporting  its  assigned  RC  units,  or  (b)  the  fail¬ 
ure  of  a  unit  to  accomplish  its  mobilization  combat  mission. 
The  checklist  items  were  written  by  aviation  subject  matter 
experts  who  are  knowledgeable  about  (a)  the  operational 
requirements  of  RC  support  facilities  in  each  of  the  func¬ 
tional  areas,  and  (b)  the  mobilization  mission  requirements 
for  RC  units. 

The  DCST,  First  U.S.  Army,  has  expressed  concern  about 
several  problems  with  (a)  the  content  of  the  checklist,  (b) 
the  manner  in  which  the  checklist  items  are  used  to  evaluate 
RC  facilities  and  units,  and  (c)  the  management  and  utiliza¬ 
tion  of  information  obtained  from  ARMS  visits.  Consequently, 
during  the  second  quarter  of  fiscal  year  1985,  the  DCST 
requested  that  the  U.S.  Army  Research  Institute  Aviation 
Research  and  Development  Activity  (ARIARDA)  provide  research 
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support  to  evaluate  and  revise  the  checklist .  Anacapa 
Sciences  began  work  on  the  project  on  3  June  1985. 


Project  Objectives 

The  general  objectives  of  the  ARMS  Checklist  research 

are  : 

•  to  perform  a  systematic  evaluation  of  the  content  of 
the  First  U.S.  Army  ARMS  Checklist, 

•  to  develop  a  set  of  recommendations  for  improving  (a) 
the  ARMS  checklist  and  (b)  the  procedures  used  to 
administer  it,  and 

•  to  develop  a  computer-based  information  management 
system  for  organizing  and  analyzing  ARMS  checklist 
data . 


Research  Approach 

A  preliminary  review  of  the  ARMS  Checklist  indicated 
that  several  deficiencies  existed  in  the  checklist  and 
administrative  procedures: 

•  The  ARMS  Checklist  is  excessively  long.  There  are 
many  items  that  may  not  be  highly  related  to  mission 
success . 

•  The  procedures  used  to  evaluate  checklist  items  and  to 
combine  ratings  from  the  various  functional  areas  into 
an  overall  rating  are  not  standardized. 

•  The  negatively  stated  item  format  does  not  allow  an 
inexperienced  evaluator  to  focus  on  the  specific 
subject  in  the  item  to  be  evaluated. 

•  The  items  are  not  listed  in  an  order  that  allows  an 
inexperienced  evaluator  to  proceed  through  the 
evaluation  steps  efficiently. 

•  The  items  are  not  identified  as  applying  specifically 
to  an  aviation  facility,  an  aviation  unit,  or  both. 

•  Many  items  are  too  general  to  be  associated  with 
observable  conditions  or  events. 

•  There  is  no  systematic  procedure  for  collating  infor¬ 
mation  about  commonly  occurring  deficiencies  observed 
across  facilities/units  during  one  year. 
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The  preliminary  review  led  to  the  development  of  three 
criteria  for  determining  if  an  item  should  be  retained  in  the 
checklist .  Specifically,  an  item  should  be  retained  in  the 
checklist  only  if  the  attribute  addressed  in  the  item  is: 

(a)  easily  detectable  during  an  ARMS  visit  (Detectability) , 

(b)  important  for  judging  the  status  of  one  of  the  functional 
areas  (Importance) ,  and  (c)  critical  for  mission  success 
(Criticality) .  A  survey  questionnaire  was  developed  to 
assess  the  extent  to  which  the  checklist  items  meet  the  three 
criteria  for  a  support  facility  and  for  a  unit.  A  separate 
version  of  the  questionnaire  was  developed  for  each  of  the 
functional  areas.  The  respondents  for  the  questionnaire  were 
aviators  and  aviation  technicians  from  First  Army  National 
Guard  and  Army  Reserve  aviation  support  facilities  and 
aviation  units. 


Project  Status 


Work  Completed 

Pretesting  of  the  questionnaires  was  completed  in 
November  1985.  Following  pretest  and  revision,  the 
questionnaires  were  mailed  to  ARNG  and  USAR  facilities  in  the 
First  Army  area.  An  average  of  23  respondents  completed  a 
questionnaire  in  each  functional  area.  Responses  to  the 
questionnaires  were  entered  into  a  data  base,  verified,  and 
analyzed.  Preliminary  results  of  the  data  analyses  were 
briefed  to  the  First  Army  DCST  in  June,  1986  and  to  staff 
members  of  the  Aviation  Division,  DCST,  First  U.S.  Army  in 
March  1987. 

The  results  indicate  that  the  deficiencies  described  in 
the  majority  of  the  checklist  items  are  detected  easily 
during  an  ARMS  evaluation  visit  and  are  moderately  important 
for  assessing  the  functional  areas  in  which  they  are 
presently  classified.  The  results  also  indicate  that  a 
facility  or  unit  with  the  deficiencies  described  in  the 
majority  of  the  checklist  items  could  support  most  aspects  of 
its  mission,  assuming  that  the  deficiencies  exist  in  isola¬ 
tion.  The  results  suggest  that  a  single  version  of  the 
checklist  needs  to  be  developed,  with  each  item  presented  as 
an  affirmative  question  instead  of  a  negative  statement  and 
clearly  annotated  to  indicate  whether  it  applies  to  a 
facility  or  to  a  unit. 

A  set  of  decision  rules  was  developed  to  aid  the  mili¬ 
tary  user  in  determining  if  items  should  be  retained  in  their 
present  form,  revised,  or  deleted  from  the  checklist  on  the 
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basis  of  the  item's  Detectability,  Importance,  and  Criti¬ 
cality  ratings.  The  decision  rules  should  be  applied  to  both 
the  facility  and  unit  checklist  item  ratings.  An  information 
data  base  was  developed  (a)  to  summarize  the  checklist  items' 
Detectability,  Importance,  and  Criticality  ratings,  and  (b) 
to  record  the  performance  of  RC  units  on  specific  checklist 
items  and  functional  areas  during  future  ARMS  evaluation 
visits.  The  data  base  was  designed  to  enable  the  First  Army 
ARMS  team  (a)  to  select  items  for  retention,  revision,  or 
deletion  using  the  decision  rules,  (b)  to  reorganize  the 
checklist  by  grouping  together  items  with  similar  content  and 
reference  publications,  (c)  to  place  the  items  in  a  sequence 
that  minimizes  evaluator  effort,  and  (d)  to  utilize  the  data 
obtained  from  future  ARMS  visits  more  effectively  (e.g.,  to 
identify  commonly  occurring  deficiencies) . 

Near  the  end  of  the  first  contract  year,  a  draft  techni¬ 
cal  report  describing  the  method  and  results  was  submitted  to 
ARIARDA  for  formal  review  (Ruffner  &  McAnulty,  1987)  .  The 
report  is  entitled  "An  Evaluation  of  the  Aviation  Resource 
Management  Survey  (ARMS)  Checklist."  ARIARDA  completed  its 
review  of  the  draft  report  and  returned  it  to  Anacapa  for 
revision  during  the  current  contract  year.  At  the  end  of  the 
current  contract  year,  the  revisions  to  the  report  are  being 
completed . 


Work  Pxo iecte.d 

Early  in  the  next  contract  year,  the  revisions  to  the 
final  report  will  be  completed  and  the  report  will  be 
submitted  to  ARIARDA. 
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DETERMINATION  OF  ARMY  RESERVE  COMPONENT 
TRAINING  REQUIREMENTS 

Dr.  John  W.  Ruffner,  Project  Director 


Background 

Anacapa  Sciences  and  U.S.  Army  Research  Institute 
Aviation  Research  and  Development  Activity  ( ARIARDA) 
personnel  have  recently  conducted  research  to  determine  the 
adequacy  of  time  allocated  to  meet  Army  National  Guard  (ARNG) 
training  requirements  and  to  determine  the  military  and 
civilian  demographic  characteristics  of  ARNG  aviators.  The 
results  of  the  ARNG  research  are  reported  in  detail  by  Szabo, 
Ruffner,  Cross,  and  Sanders  (1986)  and  Ruffner  and  Szabo 
(1986)  . 

Like  their  counterparts  in  the  ARNG,  aviators  in  the 
U.S.  Army  Reserve  (USAR)  must  meet  the  same  annual  training 
requirements  as  aviators  in  the  active  Army.  The  types  of 
information  obtained  during  the  ARNG  research  are  equally 
valuable  for  addressing  training  and  personnel  management 
issues  in  the  USAR.  Therefore,  the  First  Army  Deputy  Chief 
of  Staff  for  Training  (DCST)  requested  that  ARIARDA  and 
Anacapa  obtain  demographic  data  and  information  about  the 
adequacy  of  training  requirements  and  training  time  for  USAR 
aviators  in  the  First  Army  area. 


Project  Objectives 

The  USAR  aviation  training  requirements  research  has  six 
specific  objectives.  The  objectives  are  listed  below: 

•  determine  the  demographic  characteristics  of  the  cur¬ 
rent  First  Army  USAR  aviator  force  (e.g.,  age,  years 
of  service,  number  of  flight  hours) ; 

•  determine  the  current  career  intentions  of  First  Army 
USAR  aviators; 

•  identify  the  factors  influencing  First  Army  aviators' 
decisions  to  join,  remain  in,  or  possibly  leave  the 
USAR; 

•  determine  if  the  aviators  consider  the  amount  of  time 
allocated  for  training  to  be  adequate  for  meeting  the 
requirements ; 

•  determine  the  aviators'  willingness  to  spend  addi¬ 
tional  time  to  meet  the  training  requirements;  and 

•  identify  the  factors  that  may  affect  the  First  Army 
aviators'  ability  to  utilize  the  allocated  time  to 
meet  the  training  requirements  (i.e.,  training 
obstacles) . 
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The  research  approach  adopted  for  this  project  is 
similar  to  that  used  for  the  ARNG  project.  The  approach  is 
described  in  detail  in  Szabo  et  al .  (1986)  and  summarized  in 

Ruffner  (1987) .  Briefly,  USAR  aviators  completed  a  question¬ 
naire  during  a  weekend  drill  period  that  assessed  military 
and  civilian  demographic  variables,  adequacy  of  current 
training  requirements,  adequacy  of  the  time  allocated  to  meet 
the  requirements,  willingness  to  spend  additional  time  to 
meet  the  requirements,  and  obstacles  to  training. 


Work  began  on  the  project  in  June,  1985.  The  aviator 
questionnaire  that  was  used  in  the  ARNG  study  was  modified 
slightly  to  reflect  minor  differences  that  exist  between  the 
ARNG  and  USAR  (e.g.,  types  of  units  and  aircraft) .  In 
addition,  a  few  items  were  added  to  the  USAR  version  of  the 
questionnaire  to  obtain  information  of  interest  to  the  First 
Army  DOST  (e.g.,  extent  of  simulator  utilization).  The  USAR 
aviator  questionnaire  was  pretested  in  November  1985.  The 
feedback  obtained  during  the  pretest  visits  was  used  to 
revise  the  questionnaire. 

The  questionnaire  was  distributed  to  the  aviators  in 
March  1987.  A  total  of  139  questionnaires,  representing  56% 
of  the  USAR  aviators  in  the  First  Army  area,  was  completed 
and  returned  to  ARIARDA  between  April  1987  and  June  1987. 

Data  from  the  questionnaires  were  entered  into  a  data  base 
and  verified.  Descriptive  statistics  (i.e.,  means,  standard 
deviations,  medians,  and  percentages)  were  generated  for  each 
questionnaire  item  for  the  total  sample  and  for  subsamples  of 
commissioned  officers  and  warrant  officers. 

In  December  of  the  current  contract  year,  a  written 
summary  of  the  data  analysis  results  was  published  as  Army 
Research  Institute  Working  Paper  FR/ARDA/ASI-88-1  (Ruffner  & 
McAnulty,  1987)  .  In  general,  the  results  of  the  survey  are 
similar  to  the  results  of  the  ARNG  survey  (Szabo  et  al . , 

1986),  with  a  few  minor  differences.  The  major  results  of 
the  questionnaire  data  analysis  are  the  following: 

•  First  Army  USAR  aviators  have  somewhat  lower  experi¬ 
ence  levels  (e.g.,  flight  hours,  time  in  service, 
percent  with  combat  experience)  than  aviators  in  the 
ARNG. 


•  Similar  to  ARNG  aviators.  First  Army  USAR  aviators  are 
generally  satisfied  with  their  civilian  and  USAR  jobs 
and  generally  intend  to  stay  in  the  USAR  until  they 
are  eligible  for  at  least  a  20-year  retirement. 

Factors  that  influence  aviators  to  remain  in  the  USAR 
include  the  opportunity  to  fly,  pay,  and  retirement 
benefits.  Factors  that  may  influence  aviators  to 
leave  the  USAR  include  administrative  details  and 
politics,  unrealistic  training  goals  for  the  time  and 
resources  allocated,  and  loss  of  flight  status. 

•  First  Army  USAR  aviators  generally  rate  the  amount  of 
time  available  to  meet  continuation  training  require¬ 
ments  as  inadequate  and  are  willing  to  spend  addi¬ 
tional  paid  time  to  meet  the  requirements.  This 
finding  is  consistent  with  the  results  from  the  ARNG 
aviation  survey. 

•  Similar  to  the  ARNG  aviators,  First  Army  reserve 
aviators  judged  that  the  unavailability  of  instructor 
pilots,  an  insufficient  number  of  flight  hours,  the 
unavailability  of  training  areas,  and  the  unavail¬ 
ability  of  aircraft  are  the  major  obstacles  to  meeting 
continuation  training  requirements.  The  aviators 
judge  that  an  insufficient  amount  of  personal  time  is 
a  major  obstacle  to  meeting  additional  military 
requirements.  In  general,  the  unavailability  of 
resources  appears  to  be  a  more  serious  problem  for 
First  Army  reserve  aviators  than  it  is  for  ARNG 
aviators . 

•  First  Army  reserve  aviators  judge  that  they  need  10 
more  Additional  Flight  Training  Periods  per  year  to 
meet  the  current  training  requirements.  The  aviators 
indicate  that  they  could  afford  to  spend  approximately 
8  additional  paid  hours  each  month  meeting  the  train¬ 
ing  requirements.  No  comparable  data  are  available 
from  the  ARNG  survey. 


Work  Projected 

Completion  of  the  ARIARDA  Working  Paper  completed 
Anacapa ' s  work  on  this  project,  unless  First  Army  requests 
additional  assistance. 
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DGFS  AVIATION  AMMUNITION  AND  GUNNERY  SURVEY 
Dr.  D.  Michael  McAnulty,  Project  Director 


Background 

In  January  1987,  the  Department  of  Gunnery  and  Flight 
Systems  (DGFS)  at  the  U.S.  Army  Aviation  Center  (USAAVNC) , 
Fort  Rucker,  Alabama,  requested  that  the  U.S.  Army  Research 
Institute  Aviation  Research  and  Development  Activity 
(ARIARDA)  provide  research  support  in  conducting  an  ammuni¬ 
tion  and  gunnery  survey  of  active  Army  (AA)  and  National 
Guard  (NG)  aviation  units.  ARIARDA  agreed  to  assist  in 
designing  and  pretesting  the  survey;  to  develop  the  data 
entry,  verification,  and  analysis  programs;  to  conduct  the 
required  data  analyses;  and  to  prepare  briefing  materials  and 
technical  reports  as  required.  All  other  project  activities 
(e.g.,  administrative  coordination,  pretesting,  data  collec¬ 
tion,  data  entry)  were  to  remain  the  responsibility  of  the 
DGFS  Study  Group. 


Problem 

The  survey  research  was  designed  to  provide  an  empirical 
data  base  for  addressing  three  major  problem  areas.  First, 
the  increasing  cost  of  ammunition  and  the  competition  for 
Department  of  Defense  funds  have  created  pressure  to  reduce 
the  annual  allocation  of  ammunition  for  Army  aviation  gunnery 
training.  The  research  was  intended  to  document  the  current 
utilization  of  ammunition  in  aviation  gunnery  training,  to 
evaluate  the  success  of  the  gunnery  training,  and  to  compile 
estimates  of  the  amount  of  ammunition  required  to  maintain 
specified  Standards  in  Training  Commission  (STRAC)  readiness 
conditions.  In  addition  to  justifying  the  ammunition 
allocations,  the  resulting  data  base  was  used  by  DGFS  to 
develop  a  new  gunnery  training  manual. 

The  second  major  problem  is  the  lack  of  adequate  ranges 
on  which  to  train  and  qualify  unit  aviators.  Many  units 
complain  that  the  available  ranges  lack  the  targetry,  scoring 
devices,  and  space  required  for  effective  training.  Further¬ 
more,  limited  access  to  the  ranges  inhibits  the  gunnery 
training  and  makes  it  difficult  to  maintain  the  required 
readiness  conditions.  The  research  was  intended  to  document 
the  availability,  type,  and  utility  of  gunnery  ranges 
currently  in  use  by  Army  aviation  units. 

The  final  major  problem  is  the  lack  of  empirical  data 
about  the  utility  of  flight  simulators  for  weapon  systems 
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training.  Theoretically,  flight  simulators  can  reduce  the 
impact  of  the  first  two  problems.  That  is,  weapon  training 
can  be  conducted  without  ranges  and  without  incurring  ammuni¬ 
tion  costs.  However,  there  are  no  data  that  identify  the 
tasks  that  can  be  trained  effectively  in  simulators,  the 
amount  of  training  that  is  most  cost-effective,  or  the  extent 
that  flight  simulator  training  can  offset  the  need  for  weapon 
firing  in  the  aircraft. 

This  problem  is  compounded  by  the  single  configuration 
of  the  AH-1  attack  helicopter  flight  and  weapons  simulator 
(FWS)  that  is  used  by  unit  aviators  who  fly  different  config¬ 
urations  of  AH-1  attack  helicopters  (e.g.,  AH-1G,  AH-1S  Modi¬ 
fied,  and  AH-1S  Production  models) .  The  FWS  is  configured 
like  the  AH-1S  Fully  Modernized  helicopter.  The  survey 
research  was  designed  to  collect  information  about  the  utili¬ 
zation  of  flight  simulators  for  conducting  aerial  weapons 
training . 


Research  Methodology 

In  January  1987,  the  Commanding  General  of  the  USAAVNC 
directed  that  a  survey  of  field  unit  aviators  and  aviation 
unit  commanders  be  conducted  to  compile  the  research  data 
required  to: 

•  describe  the  current  attack  aviation  force, 

•  formulate  an  accurate  ammunition  procurement  request, 

•  evaluate  the  training  value  of  flight  simulators  for 
aerial  gunnery,  and 

•  support  revisions  to  the  Army's  aerial  gunnery  - 
training  programs. 

Despite  the  extensive  nature  of  the  project,  reporting  com¬ 
mitments  to  Department  of  the  Army  Headquarters  dictated  that 
the  survey  be  developed  and  administered,  and  the  data 
analyzed  by  December  1987. 


Survey  Development 

Survey  development  began  with  a  review  of  the  relevant 
literature,  the  then  current  aerial  gunnery  training  manual 
(Field  Manual  (FM)  1-140) ,  and  a  previous  STRAC  questionnaire 
(1987).  The  DGFS  Study  Group  then  delineated  the  Essential 
Elements  of  Analysis  (EEA)  for  the  survey.  Approximately  100 
preliminary  survey  items  were  drafted  in  the  following  ten 
topics  covered  by  the  EEA: 
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•  personal  data  about  the  respondent, 

•  military  experience  of  the  respondent, 

•  flight  experience  of  the  respondent, 

•  present  duty  assignment  of  the  respondent, 

•  suitability  of  current  gunnery  training  publications, 

•  weapon  systems  on  the  aviator's  primary  aircraft, 

•  ammunition  allocated  and  fired  during  the  1987 
training  year, 

•  utilization  of  gunnery  range  facilities, 

•  utility  of  flight  simulators  for  gunnery  training  and 
qualification,  and 

•  door  gunnery  training. 

The  prel: minary  survey  items  were  administered  to 
approximately  50  attack  helicopter  aviators  by  DGFS  person¬ 
nel.  The  results  of  this  pretest  were  used  to  produce  a 
second  draft  of  the  survey.  The  second  draft  was  divided 
into  two  forms:  Form  A  for  the  unit  aviators  and  Form  B  for 
the  unit  commanders.  Many  of  the  items  on  the  two  forms  are 
similar  in  content,  but  the  unit  aviators  were  instructed  to 
respond  to  the  items  with  respect  to  themselves  and  the  unit 
commanders  were  instructed  to  respond  to  the  items  with 
respect  to  the  entire  unit,  except  for  their  personal  data 
and  experience. 

A  pretest  of  the  second  draft  of  the  survey  was 
scheduled  but  had  to  be  cancelled  because  of  administrative 
problems  at  the  participating  installations.  DGFS  personnel, 
acting  as  aviator  subject  matter  experts,  and  Anacapa 
Sciences  personnel  reviewed  and  edited  the  final  versions  of 
the  survey  forms  and  prepared  the  required  ancillary 
materials  (e.g.,  letters  of  instruction).  Subsequently,  the 
surveys  were  approved  for  use  by  the  U.S.  Army  Soldier 
Support  Center  and  reproduced  for  administration  by  DGFS. 

Form  A  contains  68  items  divided  into  9  of  the  10  topic 
areas  listed  above;  no  questions  are  posed  to  the  unit 
aviators  about  door  gunnery.  Form  B  contains  78  items 
divided  into  all  10  topic  areas.  The  surveys  are  much  more 
comprehensive  than  the  number  of  items  indicates.  That  is, 
many  items  have  multiple  sections  or  require  a  succession  of 
responses.  Although  all  the  items  did  not  apply  to  all  the 
respondents,  there  are  472  codable  responses  on  Form  A  and 
644  codable  responses  on  Form  B.  In  addition,  both  forms 
have  several  open-ended  response  items. 


Survey  Data  Collection 


During  August  1987,  DGFS  personnel  distributed  362  com¬ 
mander  forms  and  1996  aviator  forms  to  the  AA  and  NG  units. 
The  majority  of  the  surveys  were  administered  by  installation 
points  of  contact.  The  remainder  of  the  surveys  were 
administered  by  DGFS  personnel  conducting  visits  to  field 
units . 

ARIARDA  personnel  developed  computer  programs  to  enter 
and  verify  the  survey  data.  The  survey  responses  were 
entered  and  verified  by  DGFS  and  Anacapa  personnel  as  the 
forms  were  returned  from  the  aviators  and  commanders.  Data 
collection  was  terminated  on  19  November  1987.  At  that  time, 
127  (35%)  unit  commander  forms  and  810  (41%)  unit  aviator 
forms  had  been  completed  and  returned  in  usable  form  to  DGFS 
for  processing.  In  addition,  35  commander  and  184  aviator 
forms  were  returned  either  unused  or  incomplete.  The 
percentage  of  NG  respondents  was  36.2%  for  the  unit  com¬ 
manders  and  31.9%  for  the  aviators. 

Several  meetings  were  held  with  DGFS  personnel  to 
enumerate  the  most  important  research  issues,  to  identify  the 
subset  of  survey  items  that  most  directly  address  those 
issues,  and  to  develop  a  statistical  approach  for  analyzing 
the  items.  Subsequently,  computer  programs  were  prepared  to 
analyze  the  items  that  address  the  major  problem  areas  that 
led  to  the  initiation  of  this  project.  The  data  were 
analyzed  separately  for  the  AA  and  the  NG  because  of  the 
major  differences  in  unit  mission,  types  of  aircraft  flown, 
and  training  resources  and  standards. 


Results  and  Conclusions 

The  return  rates  of  usable  surveys  were  judged  to 
provide  a  reliable  data  base  for  analysis,  although  there 
were  limitations  on  the  number  of  subsample  analyses  that 
could  be  conducted.  The  voluminous  results  of  the  survey 
data  analyses  are  summarized  in  the  seven  general  conclusions 
discussed  in  the  following  paragraphs. 

1.  The  AA  respondents  are,  on  the  average,  relatively 
young  and  inexperienced  in  their  occupational  specialty.  The 
NG  aviators  are  older  and  more  experienced  than  their  AA 
counterparts,  and  therefore  may  be  able  to  maintain  their 
skills  at  acceptable  levels  with  less  training  resources. 

The  AA  commanders  have  experience  levels  that  are  approxi¬ 
mately  equal  to  the  NG  commanders. 
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2.  The  average  AA  aviator  flew  slightly  more  than  the 
minimum  number  of  hours  required  to  maintain  his  flight 
skills  in  Fiscal  Year  1987  (FY87),  but  fired  less  than  the 
authorized  number  of  ammunition  rounds.  The  average  NG 
aviator  flew  fewer  hours  and  fired  less  ammunition  than  his 
AA  counterpart . 

3.  A  substantial  number  of  AA  and  NG  units  were  unable 
to  meet  their  training  readiness  standards  with  the  resources 
available  to  them  in  FY87 .  The  lack  of  sufficient  ammunition 
was  the  most  frequently  cited  reason  for  not  meeting  the 
standards,  but  other  resource  limitations  were  also  cited. 

4 .  Gunnery  ranges  were  not  readily  available  to  many 
units  or  did  not  have  adequate  scoring  methods.  Very  few  of 
the  ranges  were  designed  specifically  for  aviation  gunnery, 
and  most  were  shared  with  other  branches .  These  problems 
were  especially  critical  for  the  NG  units. 

5.  The  AA  aviators  used  flight  simulators  for  gunnery 
training  to  a  moderate  extent  (a  median  of  10  hours)  during 
FY87 .  Very  few  NG  units  had  access  to  simulators,  so  their 
simulator  usage  data  were  not  analyzed.  The  AA  respondents 
perceived  the  simulators  to  have  utility  for  some  types  of 
training  tasks  but  not  for  other  types.  Specif ically,  tasks 
that  were  highly  dependent  on  the  simulator  visual  system 
generally  received  lower  ratings.  The  lack  of  physical 
fidelity  between  the  AH- IS  models  and  the  FWS  was  not  judged 
to  impair  training  on  most  types  of  tasks.  AH-64  aviators 
rated  the  training  value  of  the  AH-64  Combat  Mission 
Simulator  higher  than  the  AH-1  aviators  rated  the  FWS  on  7  of 
12  types  of  training. 

6.  The  respondents  indicated  a  desire  for  standardized 
gunnery  tables  to  support  the  development  of  unit  training 
programs.  The  data  obtained  from  the  survey  respondents 
constitute  a  source  of  information  for  revising  the  FM  1-140 
tables . 

7.  The  estimates  of  ammunition  requirements  indicate 
that  the  current  STRAC  authorizations  approximate  the  minimum 
number  of  rounds  needed  to  qualify  and  sustain  the  average 
aviator's  gunnery  skills.  The  estimates  are  supported  by  the 
FY87  data  on  the  number  of  rounds  fired  (less  than 
authorized)  and  the  percentage  of  units  that  did  not  meet 
their  training  readiness  standards. 

When  considering  these  conclusions,  it  must  be  remem¬ 
bered  that  much  of  the  survey  information  is  based  on  subjec¬ 
tive  opinions  rather  than  objective  data.  Although  the 
survey  data  provide  valuable  information,  further  research, 
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including  longitudinal  surveys  and  empirical  experiments,  is 
needed  to  determine  the  amount,  frequency,  and  type  of 
training  required  to  ensure  that  U.S.  Army  and  National  Guard 
attack  helicopter  units  are  capable  of  accomplishing  their 
missions . 


Project  Status 


Work  Completed 

The  following  significant  project  activities  have  been 
completed : 

•  developing  Forms  A  and  B  of  the  survey; 

•  collecting,  processing,  and  analyzing  survey  data  from 
810  aviators  and  127  unit  commanders; 

•  preparing  briefing  materials  for  the  User's  Conference 
(November,  1987),  Department  of  the  Army  Headquarters 
briefings  (December,  1987  and  February,  1988) ,  and  the 
STRAC  conference  (June,  1988); 

•  preparing  a  technical  report  entitled  "Army  aviation 
ammunition  and  gunnery  survey:  Volume  1:  Executive 
summary"  (April,  1988);  and 

•  drafting  a  technical  report  entitled  "Army  aviation 
ammunition  and  gunnery  survey:  Volume  2:  Final 
report"  (November,  1988) . 


WQfk  PcQjg£t£.d 

The  only  remaining  project  activity  is  to  complete  the 
review  and  editing  of  the  final  report.  The  final  report 
will  be  submitted  to  ARIARDA  in  November  1988. 
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UTILIZATION/EFFECTIVENESS  OF  FLIGHT  SIMULATORS 
FOR  AVIATION  UNIT  TRAINING 
Dr.  George  L.  Kaempf,  Project  Director 


Background 

The  Army's  Synthetic  Flight  Training  System  (SFTS)  has 
been  audited  by  the  Army  Audit  Agency  (AAA)  on  two  occasions, 
first  in  1981  and  again  in  1984.  The  AAA  reports  (U.S.  Army 
Audit  Agency,  1982,  1985)  stated  that,  although  flight  simu¬ 
lators  had  reduced  training  costs  and  improved  training  at 
the  U.S.  Army  Aviation  Center  (USAAVNC) ,  the  Army  had  not 
determined  the  effects  that  the  employment  of  flight  simu¬ 
lators  may  have  on  training  in  operational  aviation  units. 
Specifically,  both  reports  admonished  the  Army  for  the  manner 
in  which  operational  tests  had  been  conducted  on  the  SFTS  and 
concluded  that  the  Army  had  not  adequately  quantified  the 
return  on  its  investment  in  flight  simulators  procured  for 
aviation  unit  training.  The  potential  returns  include  reduc¬ 
tions  in  the  number  of  aircraft  flight  hours  required  for 
training  and  increases  in  aviators'  proficiency  and  combat 
readiness  (see  Kaempf,  1986,  1987) . 

In  both  audit  reports,  the  AAA  strongly  urged  the  Army 
to  undertake  research  needed  to  quantify  the  return  on  the 
investment  in  flight  simulators  being  procured  for  opera¬ 
tional  aviation  units.  In  response  to  the  AAA  recommenda¬ 
tions,  the  U.S.  Army  Research  Institute  Aviation  Research  and 
Development  Activity  (ARIARDA)  and  Anacapa  Sciences 
researchers  have  accomplished  simulator  effectiveness 
research  under  two  separate  but  related  taskings.  The  first 
tasking  was  received  from  the  USAAVNC  Directorate  of  Training 
and  Doctrine  (DOTD)  in  June,  1984.  The  second  tasking  was 
received  from  the  Department  of  the  Army  (DA)  in  June,  1986. 


DOTD  Tasking 

The  DOTD  formally  tasked  ARIARDA  to  address  the  issues 
raised  about  the  Army's  SFTS  training  program.  Specifically, 
DOTD  requested  that  ARIARDA  conduct  research  to  answer  such 
questions  as  : 

•  What  tasks  can  best  be  trained  in  flight  simulators? 

•  What  rate  of  practice  in  flight  simulators  best 
enables  aviators  to  maintain  proficiency? 
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•  How  can  flight  simulators  be  used  to  sustain  profi¬ 
ciency  on  skills  that  are  not  routinely  practiced  in 
the  aircraft? 

•  What  effect  does  simulated  gunnery  training  as  a 
substitute  for  live-fire  training  have  on  aviator 
proficiency  and  unit  readiness? 


DA  Tasking 

In  1986,  the  Office  of  the  Deputy  Chief  of  Staff  for 
Operations  at  DA  reviewed  the  issues  concerning  the  develop¬ 
ment  of  flight  simulation  train'  g  programs  and  the  fielding 
of  the  flight  simulators.  DA  determined  that  training  effec 
tiveness  analyses  should  be  conducted  for  each  of  the  Army's 
flight  simulation  systems.  The  analyses  would  serve  as  the 
basis  for  the  development  of  effective  training  strategies 
and  programs.  Subsequently,  DA  tasked  the  Training  and 
Doctrine  Command  to  develop  and  implement,  with  the  assis¬ 
tance  of  ARIARDA,  post-fielding  training  effectiveness 
analyses  (PFTEA)  of  the  Army's  visual  flight  simulator 
systems.  ARIARDA  agreed  to  develop  and  implement  a  plan  to 
address  the  utilization  of  flight  simulators  in  operational 
environments . 


Problem 

The  Army  is  making  a  significant  investment  in  the 
development  and  acquisition  of  motion-based,  visual  flight 
simulators  for  its  rotary  wing  aircraft.  High-fidelity  simu 
lators  are  viewed  as  cost-effective  alternatives  to  flight 
training  in  the  aircraft.  Visual  flight  simulator  systems 
have  been  developed  for  the  AH-1F,  AH-64A,  CH-47D,  and  UH-60 
helicopters.  All  of  these  systems  are  in  advanced  stages  of 
deployment  to  operational  aviation  units. 


The  Army's  objective  is  to  provide  training  devices  in 
which  operational  aviators  may  sustain  their  flight  and 
tactical  skills.  However,  very  little  empirical  data  cur¬ 
rently  exist  (a)  to  demonstrate  that  flight  simulators 
effectively  and  efficiently  provide  this  type  of  training, 
and  (b)  to  guide  the  Army  in  developing  training  programs 
that  include  an  optimum  mix  of  training  conducted  in  the  air¬ 
craft  and  flight  simulator.  These  data  are  needed  to  ensure 
that  the  Army  receives  the  maximum  return  on  its  investment 
in  flight  simulation. 
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Research  Objectives 


The  overall  objective  of  this  research  area  is  to  eval¬ 
uate  the  effectiveness  of  the  U.S.  Army  flight  simulator 
systems  for  training  operational  aviators.  The  objectives  of 
the  research  projects  resulting  from  the  DOTD  and  the  DA 
taskings  are  to: 

•  develop  procedures  for  evaluating  flight  simulator 
training  effectiveness, 

•  identify  tasks  that  can  be  sustained  effectively  and 
efficiently  in  the  AH-1  Flight  and  Weapons  Simulator 
(FWS) , 

•  determine  the  effectiveness  of  the  AH1TWS  and  the  AH- 
64  Combat  Mission  Simulator  (CMS)  for  training  crew 
gunnery  tasks  and  sustaining  crew  gunnery  proficiency 
in  operational  aviation  units, 

•  provide  the  data  needed  to  develop  training  programs 
that  incorporate  a  sound  mix  of  flight  simulator  and 
aircraft  flight  training  in  operational  aviation 
units,  and 

•  identify  training  techniques  and  strategies  that  will 
enable  aviators  to  maintain  their  flight  and  tactical 
skills  in  flight  simulators. 


Re.s  e  ax ch . .  &pe>  r  a,a  ch 

A  series  of  research  projects  are  required  to  meet  the 
objectives  of  this  research  area.  During  the  current 
contract  year,  work  was  accomplished  on  five  projects.  Two 
of  the  projects  are  being  conducted  in  response  to  the  DOTD 
tasking  and  three  of  the  projects  are  being  conducted  in 
response  to  the  DA  tasking.  The  initial  approach  and  the 
specific  projects  being  conducted  in  response  to  the  DOTD  and 
DA  taskings  are  described  below. 


Response  to  the  DOTD  Tasking 

In  response  to  the  DOTD  tasking,  Anacapa  and  ARIARDA 
researchers  developed  a  research  plan  (Cross  &  Gainer,  1987) 
designed  to  answer  the  issues  raised  by  the  first  AAA  audit 
and  by  other  simulation  experts.  The  research  plan  calls  for 
a  series  of  research  efforts  at  the  USAAVNC  and  in  opera¬ 
tional  aviation  units.  The  following  two  projects  were 
initiated  to  implement  the  Cross  and  Gainer  research  plan. 

Backward  transfer  and  skill  acquisition  in  the  AH1FWS. 
Two  experiments  were  conducted  in  +-he  first  project.  In  the 
first  experiment,  AH-1  instructor  pilots  at  the  USAAVNC  who 
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were  proficient  in  the  AH-1  but  unfamiliar  with  the  FWS,  were 
evaluated  on  their  performance  of  eight  emergency  touchdown 
maneuvers  (ETMs).  Their  performance  in  the  aircraft  was 
compared  to  their  performance  in  the  simulator.  The  purpose 
of  this  experiment  was  to  determine  if  the  AH-1  aircraft  and 
the  FWS  are  interchangeable  training  devices  and  to  test  the 
utility  of  the  backward  transfer  paradigm.  In  the  second 
experiment,  four  different  groups  of  aviators  from  opera¬ 
tional  units  each  received  simulator  training  on  a  different 
set  of  five  maneuvers.  The  subjects  received  a  maximum  of 
ten  practice  trials  on  each  maneuver.  The  purpose  of  this 
experiment  was  to  estimate  how  many  practice  trials  were 
required  to  reach  a  satisfactory  level  of  proficiency  on  each 
maneuver . 

Transfer  of  training  in  the  AH1FWS  for  ETMs.  The  second 
project  was  designed  to  evaluate  the  effectiveness  of  the 
AH1FWS  for  training  five  ETMs  that  operational  aviators 
normally  are  prohibited  from  practicing  in  the  aircraft .  An 
exception  to  the  prohibition  was  granted  by  the  Army  for  the 
purpose  of  this  experiment.  Operational  unit  aviators  were 
given  checkrides  on  the  five  ETMs  in  the  AH-1  aircraft  and  in 
the  FWS,  and  then  assigned  to  a  control  group  or  an  experi¬ 
mental  group.  The  control  group  subjects  were  trained  to 
proficiency  on  the  ETMs  in  the  aircraft.  The  experimental 
group  subjects  were  trained  to  proficiency  on  the  ETMs  in  the 
AH1FWS  and  then  trained  to  proficiency  in  the  aircraft.  The 
effectiveness  of  the  simulator  training  was  tested  by  com¬ 
paring  the  two  groups  on  their  aircraft  training  performance. 


Response  to  the  DA  Tasking 

In  support  of  the  DA  tasking,  Anacapa  and  ARIARDA 
researchers  developed  a  PFTEA  plan  (ARIARDA,  1987)  that  can 
be  implemented  for  each  of  the  four  simulator  systems  being 
procured  for  the  operational  aviation  units.  The  PFTEA  plan 
proposes  to  investigate  the  effectiveness  of  each  simulator 
system  to  acquire  and  sustain  aviator  flight  skills  on  groups 
of  selected  flight  tasks.  Several  groups  of  flight  tasks 
will  be  investigated  for  each  different  simulator  system; 
each  task  group  investigated  will  require  a  separate  sample 
of  subjects.  The  task  groups  include: 

•  emergency  touchdown  maneuvers, 

•  basic  flight  tasks, 

•  flight  with  night  vision  systems,  and 

•  tactical  and  weapons  tasks. 
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Three  projects  have  been  initiated  in  response  to  the  DA 
tasking.  Each  project  is  briefly  described  in  the  following 
paragraphs . 

Training  effectiveness  analysis  of  the  AH1FWS  for 
conducting  gunnery  training.  This  project  is  designed  to 
evaluate  the  effectiveness  of  the  AH1FWS  for  sustaining 
gunnery  skills  in  operational  units.  In  this  experiment, 
rated  and  combat  qualified  AH-1  aviators  in  U.S.  Army,  Europe 
(USAREUR)  units  will  be  assigned  to  one  of  three  groups:  one 
control  and  two  experimental.  Each  subject's  initial  profi¬ 
ciency  on  crew  gunnery  tasks  will  be  evaluated  during  a 
pretest  checkride  at  the  live-fire  gunnery  range. 

Following  the  pretest  checkride,  Experimental  Group  1 
aviators  will  receive  gunnery  training  in  the  flight  simu¬ 
lator  every  month  and  Experimental  Group  2  aviators  will 
receive  gunnery  training  in  the  flight  simulator  every 
quarter.  Aviators  in  both  experimental  groups  will  continue 
to  participate  in  their  units'  normal  flying  programs  except 
for  restrictions  on  the  gunnery  tasks  that  they  can  perform 
in  the  aircraft.  Aviators  in  the  control  group  will  partici¬ 
pate  in  their  units'  normal  flying  programs  but  will  receive 
no  gunnery  training  in  the  flight  simulator  during  the 
research  period.  Following  the  12-month  period,  all  subjects 
will  be  evaluated  in  a  live-fire  crew  gunnery  training 
exercise.  The  effectiveness  of  the  AH1FWS  for  sustaining 
gunnery  skills  will  be  tested  by  comparing  the  performance  of 
the  groups  during  the  live-fire  exercise. 

Training  effectiveness  .analysis  ,.pf  the.  M64CHS  for 
conducting  crew  gunnery  initial  qualification  training.  This 
second  project  is  designed  to  evaluate  the  effectiveness  of 
the  AH64CMS  for  the  initial  training  of  aerial  gunnery  skills 
up  to  the  crew  qualification  level.  In  this  experiment,  AH- 
64A  aviators  from  the  Apache  Training  Brigade  ( ATB)  at  Fort 
Hood,  Texas,  will  be  assigned  to  two  subject  groups:  one 
experimental  and  one  control.  During  the  8-week  research 
period,  the  experimental  group  subjects  will  receive  four 
periods  in  the  AH64CMS  training  on  specific  crew  gunnery 
tasks.  The  control  group  subjects  will  receive  no  training 
in  the  AH64CMS  on  gunnery  tasks.  Concurrently,  both  groups 
will  participate  in  the  unit's  normal  flying  program. 
Following  the  training,  all  subjects  will  be  evaluated  in  a 
live-fire  crew  gunnery  training  exercise.  The  ef fectiveness 
of  the  AH 6 4 CMS  for  initial  gunnery  training  will  be  tested  by 
comparing  the  performance  of  the  groups  during  the  live-fire 
exercise  . 
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Training  effectiveness  analysis  of  the.^AH64CMS  for 
conducting  crew  gunnery  proficiency  sustainment  training. 
This  third  project  will  also  be  conducted  at  Fort  Hood, 
Texas,  and  is  designed  to  evaluate  the  effectiveness  of  the 
AH 6 4 CMS  for  sustaining  gunnery  skills  in  operational  units. 
The  experimental  design  is  the  same  as  for  the  project  that 
is  designed  to  evaluate  the  effectiveness  of  the  AH1FWS  for 
sustaining  gunnery  skills  in  operational  units. 


Project  Status 

Backward  transfer  and  skill  acquisition  in -the. . AH 1 FWS . 

All  data  collection  and  analyses  were  completed  and  described 
for  this  project  during  the  previous  contract  year  (Kaempf, 
1987) .  During  the  current  contract  year,  a  technical  report 
was  drafted  to  present  the  findings  of  the  research  project. 
In  summary,  the  results  indicate  that  AH1FWS  design  defi¬ 
ciencies  adversely  affect  pilot  training  and  performance  on 
selected  maneuvers.  Specifically,  aircraft-proficient 
aviators  required  several  training  trials  before  they  could 
perform  the  selected  maneuvers  satisfactorily  in  the  simu¬ 
lator.  The  aviators  showed  significant  improvement  on  all 
but  four  of  the  maneuvers  across  ten  training  trials.  The 
results  of  the  two  experiments  support  the  utility  of  the 
backward  transfer  paradigm  as  a  relatively  inexpensive  means 
of  estimating  forward  transfer  of  training. 

Transfer  of  training  in  the  AH1FWS  for  ETMs .  All  data 
collection  was  completed  and  described  for  this  project 
during  the  previous  contract  year  (Kaempf,  1987).  During  the 
current  contract  year,  the  data  were  analyzed  and  a  technical 
report  was  drafted  to  report  the  results  of  the  research 
project.  In  summary,  the  results  indicate  that  the  aviators 
possessed  deficient  skills  on  the  ETMs  prior  to  the  experi¬ 
ment.  Furthermore,  the  aviators  required  extensive  training 
to  reach  proficiency  in  the  FWS  but  relatively  little 
training  to  regain  proficiency  in  the  AH-1  aircraft.  Prior 
training  in  the  FWS  did  reduce  the  number  of  practice  trials 
and  the  amount  of  flight  time  required  to  reach  proficiency 
in  the  AH-1  aircraft. 

Training  effectiveness  analysis  of  the  AH1FWS  for 
conducting  gunnery  training.  This  project  was  initiated 
during  the  previous  contract  year.  Work  during  the  current 
contract  year  consisted  of  three  activities:  (a)  obtaining 

approval  to  conduct  the  research,  (b)  coordinating  support 
for  the  research  from  various  Army  agencies,  and  (c)  devel¬ 
oping  procedures  that  will  be  employed  during  the  research. 
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As  reported  previously  (Kaempf ,  1987) ,  ARIARDA  submitted 
a  proposed  research  plan  to  USAREUR  for  review  and  approval. 
The  7th  Army  Training  Command  (ATC)  assumed  the  role  of  pro¬ 
ponent  in  USAREUR  for  this  action.  In  October  1987,  the  7th 
ATC  reported  that  neither  V  Corps  nor  VII  Corps  concurred 
with  the  plan  because  the  research  would  adversely  affect  the 
units’  combat  readiness.  In  November  1987,  a  revised 
research  plan  was  submitted  to  the  7th  ATC  that : 

•  restricted  subjects  to  a  designated  crew  station  for 
the  research  period, 

•  reduced  the  number  of  experimental  groups  from  three 
to  two,  and 

•  increased  the  number  of  subjects  from  40  to  50. 

The  7th  ATC  obtained  concurrence  from  V  and  VII  Corps 
for  the  revised  research  plan  and,  on  16  May  1988,  issued  a 
message  authorizing  ARIARDA  to  begin  the  planning  and  coordi¬ 
nation  required  to  start  the  AH-1  PFTEA  research.  In  July 
1988,  the  Anacapa  project  director  relocated  to  the  Federal 
Republic  of  Germany  and  began  preparing  for  the  data  collec¬ 
tion  phase. 

Training  effectiveness  analysis  of the  AH64CMS  for 
conducting  crew _ .gunnery .-iRi.tia.l— qua li£ i ca t.iojD -t.ra.in ing  and 
training  effectiveness,  analysis  _of  the  AH .6,4 CMS  -fox,  conducting 
crew  gunnery. „ proficiency  .sustainment  training .  in  March 
1988,  the  Directorate  of  Gunnery  and  Flight  Systems  (DGFS ) 
officials  requested  that  ARIARDA  draft  a  research  plan  for 
evaluating  the  effectiveness  of  the  AH 6 4 CMS  for  training 
gunnery  tasks.  DGFS  directed  that  the  research  (a)  identify, 
on  a  task-by-task  basis,  the  effectiveness  of  the  CMS  for 
training  crew  gunnery  tasks,  (b)  focus  on  the  training  of 
flight  crews  assigned  to  operational  aviation  units,  and  (c) 
provide  data  that  will  support  future  requests  for  training 
ammunition . 

The  Anacapa  project  director  drafted  the  research  plan 
and  ARIARDA  briefed  the  proposal  to  DGFS  and  USAAVNC  offi¬ 
cials.  Because  of  the  concentration  of  AH-64  resources  in 
the  ATB  and  the  6th  Cavalry  Brigade  -  Air  Combat  (CBAC) , 
ARIARDA  identified  Fort  Hood,  Texas,  as  the  optimal  site  for 
conducting  the  research.  DGFS  and  USAAVNC  officials  approved 
the  research  plan.  In  subsequent  briefings  at  Fort  Hood,  the 
ATB  concurred  with  the  need  for  the  proposed  research  but 
expressed  reluctance  to  commit  resources  to  support  the 
research.  The  6th  CBAC  committed  full  support  for  the 
research  project . 


During  June  1988,  final  project  planning  was  conducted 
and  data  collection  was  scheduled  to  begin  on  5  July  in  the 


ATB  and  on  8  August  in  the  6th  CBAC.  The  6th  CBAC  remained 
fully  supportive  of  the  research  project,  but  the  issues  of 
insufficient  range  time  and  insufficient  ammunition  had  not 
been  resolved.  Coordination  with  the  III  Corps  agencies  at 
Fort  Hood  and  the  Standards  in  Training  Commission  Program 
Directorate  at  Fort  Eustis,  Virginia,  resolved  both  of  these 
issues.  ARIARDA  submitted  a  written  research  plan  for 
approval  by  III  Corps  and  requested  that  DA  task  III  Corps  to 
support  the  ARIARDA  research  project.  DA  tasked  III  Corps  to 
support  the  research  on  1  July  1988. 

Both  projects  were  rescheduled  to  start  in  August  1988; 
however,  additional  coordination  problems  with  the  support 
units  at  Fort  Hood  caused  the  skill  acquisition  project  to  be 
postponed  until  December  1988.  The  live-fire  pretest  for  the 
skill  sustainment  project  began  on  6  August  1988,  but  was 
suspended  on  7  August  1988  because  of  a  malfunction  in  the 
automated  area  weapons  scoring  system.  The  skill  sustainment 
project  was  rescheduled  for  February  1989. 


Work  ...Ei;  ejected 

Backward  ..transfex  and,. .skill  acquisition,  ia  -the  .AHlf  WS.  ■ 

During  the  next  contract  year,  the  draft  technical  report  for 
this  project  will  be  edited  prior  to  submission  to  ARIARDA 
for  review.  The  report  (Kaempf,  Cross,  &  Blackwell,  1988) 
will  be  submitted  to  ARIARDA  early  in  the  next  contract  year. 
Unless  revisions  are  required,  delivery  of  the  technical 
report  will  complete  all  activities  under  this  project. 

Transfer  of  training  in  the  AH1FWS  for  ETMs .  During  the 
next  contract  year,  the  draft  technical  report  for  this 
project  will  be  edited  prior  to  submission  to  ARIARDA  for 
review.  The  report  (Kaempf  &  Blackwell,  1988)  will  be  sub¬ 
mitted  to  ARIARDA  early  in  the  next  contract  year.  Unless 
revisions  are  required,  delivery  of  the  technical  report  will 
complete  all  activities  under  this  project. 

Training  effectiveness  analysis  of  the  AH1FWS  for 
conducting  gunnery  training.  During  the  next  contract  year, 
the  experimental  training  and  data  collection  will  begin. 

The  initial  live-fire  pretest  exercise  is  tentatively 
scheduled  to  begin  in  late  November  or  early  December  of 
1988.  Simulator  training  will  begin  in  January  1989.  If 
this  project  proceeds  as  scheduled,  the  final  live-fire 
exercises  will  be  conducted  during  January  1990. 
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learning  .effect 3.vene£^_.aaaIy5J.s_Qi  the_Ah64CMS  for 

conducting  crew  gunnery  initial  qualification  training. 

During  the  next  contract  year,  the  experimental  training  and 
data  collection  will  begin  for  this  project.  Simulator 
training  is  scheduled  to  begin  in  December  1988  and  end  in 
July  1989.  Live-fire  crew  gunnery  training  is  scheduled  to 
begin  in  February  1989  and  to  be  completed  in  June  1989. 
Subsequently,  the  data  will  be  processed  and  analyzed,  and  a 
technical  report  will  be  drafted  to  present  the  project 
results . 

Training  effectiveness _ analysis_nf  the.  AH64.CMS  for 
conducting  crew  .gunnery  proficiency  sustainment  training. 
During  the  next  contract  year,  data  collection  will  also 
begin  on  this  project .  Simulator  training  is  scheduled  to 
begin  for  the  experimental  groups  in  March  1989  and  to  end  in 
February  1990.  All  subjects  will  undergo  their  units’  normal 
flying  programs.  The  live-fire  training  exercises  are 
scheduled  to  begin  in  February  1989  and  again  in  February 
1990. 
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DEVELOPMENT  AND  EVALUATION  OF  THE 
AH- 64  DISPLAY  TRAINING  MODULE 

Dr.  John  W.  Ruffner,  Project  Director 


Background 

The  AH-64  attack  helicopter  is  a  two-crewmember  aircraft 
designed  to  fly  nap-of-the-earth  missions  to  detect,  engage, 
and  destroy  enemy  armor  during  day  or  night  and  in  all 
weather  conditions.  To  provide  this  capability,  the  AH-64  is 
equipped  with  several  complex  flight  and  weapons  delivery 
systems.  The  successful  operation  of  some  of  these  systems 
requires  that  the  pilot  and  copilot/gunner  (CPG)  be  able  to 
observe,  interpret,  and  integrate  both  visual  imagery  and 
symbolic  information  presented  on  visual  displays. 

The  AH-64  visual  display  systems  that  provide 
information  to  the  pilot  and/or  the  CPG  are  the  Target 
Acquisition  and  Detection  System  (TADS) ,  the  Pilot  Night 
Vision  System  (PNVS) ,  and  the  Fire  Control  Symbol  Generator. 
The  TADS  is  used  by  the  CPG  for  target  search,  detection, 
recognition,  and  designation.  Three  sensors,  the  forward- 
looking  infrared  (FLIR)  system,  the  day  television  viewing 
system,  and  the  direct  view  optics  system,  provide  the  CPG 
with  visual  information  to  detect  and  engage  targets  at 
standoff  ranges  during  day  or  night  and  in  adverse  weather 
conditions.  The  PNVS  provides  FLIR  imagery  that  enables  the 
pilot  to  fly  the  aircraft  at  night  and  during  degraded 
visibility  conditions.  The  fire  control  symbol  generator 
superimposes  flight  and  weapons  symbology  on  the  imagery 
displayed  by  the  TADS  and  PNVS. 

Visual  imagery  and  symbology  from  the  TADS  and  the  PNVS 
can  be  presented  to  either  crewmember  on  the  panel-mounted 
cathode  ray  tube  (CRT)  displays  or  through  the  Helmet-Mounted 
Display  (HMD).  The  HMD  provides  a  30°  (vertical)  by  40° 
(horizontal)  field  of  view  to  the  crewmember  on  a  1-inch 
diameter  CRT  attached  to  the  helmet.  The  HMD  enables  the 
crewmember  to  cross-check  flight  and  weapons  information 
while  simultaneously  viewing  the  external  visual  scene. 

The  27  different  symbols  in  the  PNVS  flight  symbol  set 
can  be  presented  to  assist  the  crewmember  in  flying  the 
aircraft.  Slightly  different  subsets  of  the  symbols  are 
presented  during  hover,  transition,  cruise,  and  bob-up  modes. 
The  17  different  symbols  in  the  weapons  symbol  set  can  be 
presented  to  assist  the  crewmember  during  the  operation  of 
the  weapons  system.  Fourteen  symbols  are  common  to  both  the 
flight  and  weapons  symbol  sets.  The  symbols  vary  in  size. 
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shape,  location,  and  the  manner  in  which  they  represent  the 
status  of  the  aircraft  or  weapons  system  elements .  Informa¬ 
tion  about  an  aircraft  or  weapons  system  may  be  represented 
by  changes  in  symbol  size,  position,  or  rate  of  movement. 

The  number  of  symbols  displayed  at  any  given  time  depends  on 
the  nature  of  the  flight  or  weapons  tasks  and  the  selected 
symbology  mode . 


Nee.d 

To  become  fully  qualified  in  the  AH-64  attack 
helicopter,  a  student  aviator  must  learn  to  recognize, 
understand,  interpret,  and  integrate  the  symbology  presented 
on  the  helicopter's  visual  displays.  During  the  AH-64 
Aircraft  Qualification  Course  (AQC) ,  student  aviators  are 
taught  to  use  the  symbology  through  classroom  lectures, 
videotape  presentations,  self-study  handouts,  and  technical 
manuals  containing  static  diagrams  of  the  symbology. 
Opportunities  for  additional  practice  with  the  display 
symbology  are  available  on  three  training  devices:  (a)  the 

TADS  Selected  Task  Trainer  (TSTT) ,  (b)  the  Cockpit  Weapons 

Emergency  Procedures  Trainer  (CWEPT) ,  and  (c)  the  Combat 
Mission  Simulator  (CMS) . 

The  TSTT  is  a  part-task  trainer  designed  to  support  (a) 
initial  CPG  qualification  and  refresher  training  in  the  AH- 
64,  and  (b)  TADS  skill  sustainment  during  mission  and 
continuation  training  in  operational  aviation  units.  It 
provides  practice  only  with  weapons  symbology.  The  CWEPT  is 
a  full-scale  crew  station  procedures  trainer.  It  is  designed 
to  provide  training  to  the  pilot  and  the  CPG  in  both  normal 
and  emergency  flight  procedures  and  avionics  equipment 
operation.  The  CMS  is  a  six  degree-of-f reedom,  motion-based 
simulator  designed  to  simulate  the  flight  and  weapons 
capabilities  of  the  AH-64  aircraft.  It  is  currently  used  to 
provide  training  in  combat  mission  scenarios  during  the 
Combat  Skills  phase  of  the  AH-64  AQC  and  during  operational 
aviation  unit  training. 

The  training  design  features  of  the  TSTT,  CWEPT,  and  CMS 
do  not  include  training  on  basic  symbology  identification  and 
interpretation.  Students  assigned  to  training  lessons  on 
these  devices  are  assumed  to  be  familiar  with  flight  and 
weapons  symbology.  However,  TSTT,  CWEPT,  and  CMS  instructors 
expend  an  inordinate  amount  of  time  training  basic  symbology 
skills  in  the  respective  devices.  Furthermore,  students 
training  in  the  TSTT,  CWEPT,  or  CMS  typically  do  not  have 
opportunities  to  use  the  AH-64  display  symbology  under  the 
full  range  of  missions,  modes,  weapons,  system  options,  and 
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system  failures.  A  device  that  provides  specialized  training 
on  basic  symbology  identification  and  interpretation  would 
improve  the  efficiency  of  TSTT,  CWEPT,  and  CMS  instruction. 


Project  Objectives 

The  Training  and  Doctrine  Command  System  Manager  for  the 
AH-64  requested  that  the  Army  Research  Institute  Aviation 
Research  and  Development  Activity  (ARIARDA)  develop  and 
empirically  evaluate  a  module  to  train  AH-64  student  aviators 
to  understand  and  interpret  flight  and  weapons  symbology. 

ARIARDA  established  the  following  six  specific  design 
objectives  for  the  training  module: 

•  be  designed  in  a  self-instructional  format  (i.e.,  not 
require  an  instructor  to  operate  the  trainer) ; 

•  be  designed  to  provide  training  in  a  classroom 
setting; 

•  be  designed  to  train  symbology  for  the  full  range  of 
aircraft  mission  and  weapon  system  options; 

•  be  capable  of  storing  performance  data  and  providing 
one  or  more  performance  indexes  after  each  training 
exercise ; 

•  be  capable  of  providing  immediate  feedback  and 
remedial  instruction  when  errors  occur; 

•  be  suitable  for  both  initial  skill  acquisition  in  an 
institutional  training  setting  and  skill  sustainment 
training  in  an  operational  unit  training  setting;  and 

•  be  flexible  enough  to  allow  revisions  resulting  from 
(a)  design  changes  in  the  aircraft,  (b)  design  changes 
in  the  avionics  system,  or  (c)  deficiencies  in  the 
training  module  revealed  by  formal  evaluation  and 
feedback  from  the  user. 

In  addition,  the  training  module  should  be  designed  to 
augment  but  not  replace  existing  training  devices.  Finally, 
it  should  not  require  the  fabrication  and  use  of  mockups  or 
other  costly  training  aids. 


Research  Approach 

Ten  research  tasks  are  required  to  accomplish  the 
project  objectives.  The  research  tasks  are  the  following: 

•  review  the  relevant  literature, 

•  identify  the  flight  and  weapons  tasks  requiring  the 
use  of  symbology, 

•  interview  subject  matter  experts  (SMEs) , 
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•  identify  the  capabilities  and  limitations  of  existing 
training  devices, 

•  define  the  scope  of  training, 

•  define  the  training  approaches  and  settings, 

•  define  the  terminal  learning  objectives, 

•  identify  and  evaluate  the  training  media, 

•  design  the  prototype  training  module,  and 

•  evaluate  the  prototype  training  module. 


Project  Status 


Work  Completed 

Work  began  on  the  project  in  December  1986.  As  a  result 
of  discussions  with  SMEs,  it  was  learned  that  deficiencies  in 
symbology  usage  exist  in  the  following  areas: 

•  recognizing  and  interpreting  the  meaning  of  symbols 
presented  alone  and  in  the  context  of  other  symbols, 

•  interpreting  the  meaning  of  symbology  movement, 

•  correctly  associating  switch  actions  and  control 
movements  with  static  or  dynamic  symbology,  and 

•  selectively  attending  to  and  interpreting  display 
symbology  when  the  symbology  is  superimposed  on  a 
dynamic  external  visual  scene. 

A  decision  was  made  to  design  the  display  symbology 
training  module  in  two  parts.  Part  I  will  cover  basic 
symbology  usage  skills  and  will  address  the  first  two 
deficiency  areas  cited  above.  It  is  organized  into  five 
self-contained  lessons  that  present  instructional  material 
and  quizzes  on  the  following  groups  of  symbols: 

•  position/movemert  symbols, 

•  attitude/altitude  symbols, 

•  heading/navigation  symbols, 

•  cueing/reference  symbols,  and 

•  weapon  delivery  symbols. 

Part  II  will  cover  advanced  symbology  usage  skills  and  will 
address  the  last  two  deficiency  areas.  The  specific  content 
and  organization  of  Part  II  had  not  been  determined  by  the 
end  of  the  current  contract  year. 

Work  on  the  training  module  was  suspended  at  the  end  of 
December  1987  because  of  funding  constraints  and  other 
project's  priorities.  The  portions  of  the  training  module 
that  had  been  developed  by  that  time  are  described  in  the 
following  paragraphs. 
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The  training  module  consists  of  (a)  an  introductory 
section,  (b)  a  help  system  that  is  accessible  from  any  part 
of  the  training  module,  and  (c)  tutorials  and  quizzes  for 
Lesson  1  (Position/Movement  Symbols)  and  Lesson  2  (Attitude/ 
Altitude  Symbols) .  The  tutorials  contain  static  frames  and 
animated  sequences,  as  appropriate. 

The  help  system  includes  the  following  features: 

•  an  overview  of  the  entire  symbology  training  module; 

•  an  explanation  of  the  types  of  help  provided  (e.g., 
procedural  and  content) ; 

•  a  list  of  the  contents  of  the  five  lessons  comprising 
the  training  module; 

•  an  acronym  glossary; 

•  a  symbol  dictionary;  and 

•  a  symbology  mode  dictionary. 

The  symbol  dictionary  consists  of  27  screens,  one  screen 
for  each  symbol.  Each  screen  contains: 

•  a  graphic  representation  of  the  symbol; 

•  the  symbol's  name; 

•  the  purpose  of  the  symbol; 

•  the  modes  in  which  the  symbol  appears  (e.g.,  hover, 
bob-up,  cruise,  and  transition) ; 

•  failure  actions  associated  with  the  symbol;  and 

•  the  tutorial  lesson (s)  in  which  the  symbol  is 
presented . 

The  symbology  mode  dictionary  consists  of  four  screens 
showing,  for  each  symbology  mode,  a  complete  set  of  the 
symbols  that  can  appear  in  that  mode  and  the  purpose  of  the 
mode.  Symbols  that  are  specific  to  one  of  the  modes  are 
shown  in  a  different  color  than  the  symbols  that  are  specific 
to  the  other  three  modes. 

The  storyboards  for  the  Lesson  3  (Heading/Navigation 
Symbols)  tutorial  and  quiz  have  been  developed,  but  have  not 
been  coded  on  the  computer.  A  draft  version  of  the 
storyboards  for  the  Lesson  4  (Cueing/Reference  Symbols) 
tutorial  and  quiz  have  been  completed  and  reviewed  by  members 
of  the  Anacapa  Sciences  staff,  but  have  not  been  revised  and 
coded.  No  work  has  started  on  Lesson  5  (Weapons  Usage 
Symbols)  or  on  Part  II  of  the  training  module. 

The  training  module  is  written  in  compiled  BASIC 
language  and  is  designed  to  run  on  a  Zenith  PC  AT-compatible 
microcomputer  equipped  with  one  megabyte  of  random  access 
memory,  a  20-megabyte  internal  hard  disc,  an  enhanced 
graphics  adapter  (EGA) ,  and  a  high  resolution  EGA  color 
monitor . 
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The  portions  of  the  display  symbology  training  module 
that  were  completed  by  the  end  of  December  1987  were 
demonstrated  for  the  ARIARDA  Contracting  Officer’s  Technical 
Representative  during  the  first  week  of  January  1988.  No 
additional  work  has  been  performed  on  the  project  since  that 
time . 


At  the  end  of  the  current  contract  year,  ARIARDA 
directed  Anacapa  to  terminate  the  project.  ARIARDA  requested 
that  Anacapa  submit  as  project  deliverables  a  brief  project 
summary,  storyboards,  and  computer  programs  that  have  been 
developed . 


Work  Projected 

A  brief  project  summary,  storyboards,  and  computer 
programs  will  be  submitted  to  ARIARDA  as  project  deliverables 
early  in  the  next  contract  year. 
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DEVELOPMENT  OF  THE  BASIC  MAP  INTERPRETATION  AND  TERRAIN 

ANALYSIS  COURSE  (MITAC) 

Dr.  Dudley  J.  Terrell,  Project  Director 


Background 

In  the  modern  battlefield,  Army  aviators  will  be  forced 
to  fly  at  extremely  low  altitudes  to  avoid  detection  by  enemy 
electronic  sensors.  Because  of  the  need  to  maintain  obstacle 
clearance  while  remaining  masked  by  terrain  features,  low 
altitude  flight  requires  constant  vigilance  outside  the  cock¬ 
pit.  Momentary  shifts  of  attention  to  displays,  switches, 
and  maps  inside  the  cockpit  must  be  executed  rapidly  and 
efficiently.  Consequently,  low  altitude  navigation  requires 
superior  skill  in  map  interpretation  and  terrain  analysis. 

To  remain  geographically  oriented  at  all  times,  a'  pilot  must 
be  capable  of  gleaning  crucial  map  information  during  brief 
glances  inside  the  cockpit  and  associating  that  information 
with  the  rapidly  changing  terrain  outside  the  cockpit . 

Traditional  methods  of  low  altitude  navigation  training 
have  been  unsatisfactory  (Fineberg,  Meister,  &  Farrell,  1978; 
Gainer  &  Sullivan,  1976;  McGrath,  1976).  Therefore,  the  U.S. 
Army  Research  Institute  Aviation  Research  and  Development 
Activity  (ARIARDA)  has  conducted  research  to  address  the  low 
altitude  navigation  training  deficiency.  In  1976  Anacapa 
Sciences,  under  contract  to  ARIARDA,  developed  the  Map 
Interpretation  and  Terrain  Analysis  Course  (MITAC) .  The 
MITAC  comprised  numerous  photographic  slides  and  motion 
picture  films  of  terrain  features  and  map  segments  designed 
to  teach  low  altitude  navigation  skills  to  helicopter  pilots 
in  a  classroom  format.  Subsequently,  the  course  was  revised 
to  an  individualized  training  format  utilizing  the  equipment 
of  the  Beseler  Cue/See  system  (Harman,  1978) .  Holman  (1978a, 
1978b)  demonstrated  the  effectiveness  of  this  course  by 
showing  that  MITAC-trained  student  pilots  and  enlisted  aerial 
observers  navigated  at  twice  the  speed  and  with  one-third  of 
the  errors  committed  by  traditionally  trained  aviators. 

Thirteen  additional  cinematic  exercises  were  developed 
to  provide  supplemental  training  in  map  interpretation  and 
terrain  analysis  over  a  wider  range  of  geographic  regions  and 
climates  (Kelley,  1979).  Each  of  these  exercises  consists  of 
a  film  taken  from  the  front  window  of  a  helicopter  flying  a 
route  at  low  altitude.  The  exercises  include  flights  over 
various  geographic  regions  (e.g.,  Kentucky,  Idaho,  Arizona, 
and  Germany)  with  both  snow-covered  terrain  and  summer 
foliage.  These  supplemental  exercises,  termed  the  Advanced 
MITAC,  have  been  upgraded  to  a  computer-based  interactive 
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videodisc  format  (Miles  &  LaPointe,  1986) .  Terrell  (1988) 
demonstrated  the  training  effectiveness  of  the  computer-based 
Advanced  MITAC  by  comparing  the  contour-level  navigation 
performance  of  student  pilots  who  had  received  the  supple¬ 
mental  training  with  those  who  had  not.  A  significantly 
greater  proportion  of  Advanced  MITAC  students  than  control 
(no  MITAC)  students  performed  perfectly  (i.e.,  no  deviations 
from  the  prescribed  route)  during  the  post-training 
navigation  test. 


Need 

Conversion  of  the  Advanced  MITAC  exercises  to  an  inter¬ 
active  videodisc  format  has  resulted  in  an  easy-to-use  and 
effective  part-task  trainer.  However,  the  material  and 
equipment  for  the  original  "Basic"  MITAC  (e.g.,  35-mm  slides, 
booklets,  projectors,  tape  players,  etc.)  are  difficult  to 
use  and  are  unsuitable  for  computer-based  training.  A 
computer-based  course  that  presents  basic  principles  of  map 
interpretation  is  needed  to  serve  as  a  prerequisite  course  to 
the  Advanced  MITAC  and  for  use  in  the  ARIARDA  research 
program . 


Project  Objectives 

The  goal  of  this  project  is  to  develop  a  videodisc/ 
computer-based  Basic  MITAC.  Specifically,  the  objectives  are 
to : 

•  develop  the  Basic  MITAC  videodiscs,  and 

•  design  experimental  courseware  for  the  Basic  MITAC. 


Development  Approach 

Development  will  proceed  in  two  phases.  In  Phase  1,  the 
material  from  former  versions  of  Basic  MITAC  will  be  compiled 
and  upgraded  for  videodisc  development.  In  Phase  2,  instruc¬ 
tional  strategies  will  be  planned  and  software  written  to 
implement  the  strategies.  Training  effectiveness  evaluations 
of  the  Basic  MITAC  instructional  strategies  will  be  conducted 
under  a  separate  project  title  (see  Training  Effectiveness  of 
Aviation  Part-Task  Trainers,  pp .  103-108  of  this  report). 


94 


Phase  1,  development  of  the  Basic  MITAC  videodiscs,  was 
completed  in  three  stages:  production,  post-production,  and 
duplication.  A  summary  report  of  the  Phase  1  activities  was 
delivered  to  ARIARDA  for  review  (Terrell  &  Miles,  1988)  . 

Production .  The  production  stage  was  accomplished  by 
Anacapa  researchers.  The  primary  activities  accomplished 
during  the  production  stage  were: 

•  composition  of  a  narrative  for  the  Basic  MITAC, 

•  selection  of  video  material  to  supplement  the 
narrative,  and 

•  development  of  a  script  containing  the  narrative  and 
instructions  for  taping  the  narrative  and  video 
material . 

The  narrative  was  developed  from  (a)  a  set  of  illus¬ 
trated  lectures  for  Marine  infantrymen  (Cross  &  Rugge,  1982) , 
(b)  a  classroom  handout  on  low  altitude  map  interpretation 
for  Army  aviation  students  (McGrath,  1975),  and  (c)  the 
Defense  Mapping  Agency  guidelines  for  drawing  1 : 50, 000-scale 
topographic  maps.  Many  of  the  slides,  maps,  and  charts  from 
the  original  MITAC  (i.e.,  the  Beseler  Cue/See  version)  were 
selected  for  the  Basic  MITAC.  Additional  slides  were  made  by 
photographing  various  geographic  features  in  the  Fort  Rucker, 
Alabama,  training  area.  New  and  updated  maps  were  matched  to 
the  features  on  the  slides.  Computer-generated  graphics  and 
animation  were  designed  to  supplement  portions  of  the  narra¬ 
tive.  Finally,  the  video  material  was  matched  to  the  appro¬ 
priate  segments  in  the  narrative,  and  instructions  were 
written  about  the  sequence  and  timing  of  audio  and  video 
material . 

Post-production .  The  post-production  was  accomplished 
under  subcontract  by  Video  Technics,  Inc.,  of  Atlanta, 
Georgia.  The  primary  activities  accomplished  during  the 
post-production  stage  were: 

•  recording  the  narrative  on  audio  tape, 

•  generating  computer  graphics  and  animation, 

•  digitizing  video  material, 

•  editing  video  material  on  3/4-inch  videotape,  and 

•  dubbing  audio  and  video  on  1-inch  master  videotape. 

After  the  narrative  was  recorded,  the  audio  tape  was 
time-coded  according  to  the  instructions.  Graphics  artists 
generated  charts  and  animation  segments  using  2-dimensional 


and  3-dimensional  computer  graphics  packages.  The  slide  and 
map  images  were  digitized  with  a  digitizing  camera  and  stored 
in  computer  files.  Then  the  digitized  images,  computer¬ 
generated  charts,  and  animation  were  transferred  to  3/4-inch 
tape  according  to  the  instructions  and  edited  against  the 
time-coded  audio  tape.  The  audio  and  video  material  were 
combined  when  the  1-inch  master  tape  was  dubbed. 

Duplication .  Duplication  was  accomplished  by  Optical 
Recording  Project/3M  in  St.  Paul,  Minnesota.  The  primary 
activities  during  the  duplication  stage  were: 

•  generating  a  master  videodisc  from  the  1-inch  master 
tape,  and 

•  producing  videodisc  copies  from  the  master  videodisc. 

The  generation  and  the  duplication  of  the  master  videodiscs 
were  accomplished  according  to  the  standard  procedures  of 
Optical  Recording  Project/3M. 


Work  Projected 

In  Phase  2,  instructional  strategies  will  be  developed, 
and  a  computer  will  be  programmed  to  provide  interactive 
training  using  the  videodiscs.  The  computer  programs  will 
facilitate  the  experimental  evaluation  of  several  strategies 
and  tactics  of  computer-based  instruction.  For  example, 
experimental  courseware  will  be  developed  to  examine: 

•  different  computer-based  branching  routines  for 
remediating  knowledge  or  skill  deficiencies, 

•  different  methods  for  presenting  drills  and  tutorials, 

•  the  differential  effects  of  full-motion  and  still 
graphics , 

•  the  differential  effects  of  digital  and  analog 
imagery,  and 

•  the  effects  of  audio  narrative  on  knowledge  or  skill 
acquisition . 

Finally,  the  Basic  MITAC  materials  will  be  turned  over 
to  another  project  for  training  effectiveness  evaluation  (see 
Training  Effectiveness  of  Aviation  Part-Task  Trainers,  pp . 
103-108  of  this  report) . 
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SURVEY  OF  RESEARCH  IN  COMPUTER-BASED 
INSTRUCTIONAL  STRATEGIES 

Dr.  Dudley  J.  Terrell,  Project  Director 


Background 

Experimental  research  concerning  computer-based  instruc¬ 
tional  strategies  is  a  recent  development  from  two  diverse 
areas:  learning  psychology  and  computer  technology  (for  a 

review,  see  Eberts  &  Brock,  1987).  Psychology  became 
involved  through  the  application  of  learning  research  to  the 
design  of  early  teaching  machines  (Benjamin,  1988) .  A 
growing  interest  in  the  application  of  computer  technology  to 
education  has  resulted  in  a  new  field  of  research  and 
development  called  instructional  design  (Gagne,  Briggs,  & 
Wager,  1988) .  Developments  in  computer-based  instruction 
promise  to  increase  training  effectiveness  by  capitalizing  on 
the  efficiency  of  the  individual  learning  process  while 
reducing  human  instructor  time.  Recognizing  this  promise, 
the  Department  of  Defense  has  promoted  computer-based 
training  research  through  the  efforts  of  the  Army  Research 
Institute,  the  Army  Project  Manager  for  Training  Devices,  the 
Air  Force  Human  Resources  Laboratory,  the  Navy  Personnel 
Research  and  Development  Center,  and  the  Naval  Training 
Equipment  Center  (Dallman  et  al . ,  1983;  O'Neil  &  Evans, 

1983)  . 


iNead 

Much  of  the  current  computer-based  training  research 
consists  of  piecemeal  attempts  to  apply  recent  hardware  and 
software  developments  (e.g.,  interactive  videodisc,  high¬ 
speed  personal  computers,  computer  graphics)  to  existing 
training  programs.  Lacking  a  comprehensive  plan  for  system¬ 
atic  experimentation,  such  research  solves  immediate  training 
problems,  but  fails  to  provide  information  about  the  general 
utility  of  training  technology  and  the  most  cost-effective 
methods  for  addressing  future  training  needs .  An  efficient 
program  of  computer-based  training  research  will  require  a 
comprehensive  plan  addressing  the  following  issues. 

•  What  is  the  best  method  of  identifying  the  critical 
learning  objectives  for  the  proposed  training  program? 

•  What  are  the  underlying  learning  principles  that  can 
be  applied  during  the  design  and  development  of  the 
proposed  training  program? 
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What  are  the  best  media  for  presenting  the  proposed 
training  program? 


Scientists  at  the  U.S.  Army  Research  Institute  Aviation 
Research  and  Development  Activity  (ARIAJRDA)  are  developing  a 
plan  to  integrate  existing  computer-based  training  effective¬ 
ness  research  projects  and  to  forecast  the  development  of 
future  projects.  The  development  of  this  comprehensive 
research  plan  requires  a  survey  of  past  research  on  learning 
principles  and  their  application  during  the  design  of 
computer-based  training  programs. 


Project  Objectives 

The  goal  of  this  project  is  to  generate  research  ques¬ 
tions  that  can  be  addressed  in  ARIARDA  computer-based 
training  ef f ect iveness  projects.  Specifically,  the 
objectives  are  to: 

•  survey  the  research  in  computer-based  instructional 
strategies,  and 

•  organize  the  literature  in  a  manner  that  will  facili¬ 
tate  the  development  of  a  research  plan  for  computer- 
based  aviation  training. 


Research  Approach* 

Under  this  project,  researchers  at  Anacapa  Sciences  are 
conducting  a  comprehensive  survey  of  research  on  the  applica¬ 
tion  of  learning  principles  to  computer-based  instructional 
design.  The  project  will  be  completed  in  three  phases. 

During  Phase  1,  a  data  base  will  be  compiled  of  published 
research  in  computer-based  instructional  strategies.  During 
Phase  2,  the  research  results  will  be  evaluated  for  their 
immediate  applicability  to  computer-based  instructional 
design.  During  Phase  3,  a  list  of  questions  requiring 
further  research  in  computer-based  trailing  will  be 
generated . 


Project  Status 


Work  Completed 

Phase  1  has  been  completed.  The  Psychlnfo,  Defense 
Technical  Information  Center,  Educational  Resources 
Information  Center,  and  Dissertation  Abstracts  International 
data  bases  were  searched  using  the  key  words  computer-based 


and 


These 


searches  produced  over  185  citations. 

An  in-house  data  base  format  was  designed  using  Micro¬ 
soft  File  software  and  a  Macintosh  computer.  Anacapa 
researchers  reviewed  each  article.  For  each  article,  they 
composed  the  summary  information  needed  to  complete  the 
following  entries  in  the  standardized  data  base: 

•  official  citation, 

•  type  of  document, 

•  problem  or  background, 

•  subjects, 

•  materials  or  apparatus, 

•  procedure, 

•  results, 

•  conclusions,  and 

•  reviewer's  comments  or  critique. 


Work  Projected 

During  Phase  2,  the  research  reports  will  be  reviewed  in 
more  detail.  The  applicability  of  the  research  results  to 
computer-based  aviation  training  will  be  evaluated.  Theoret¬ 
ical  papers,  position  papers,  and  courseware  design  guide¬ 
lines  will  also  be  evaluated.  These  reviews  will  reveal 
issues  in  computer-based  training  that  require  further 
research . 

During  Phase  3,  research  questions  will  be  formulated  to 
address  the  issues  identified  in  Phase  2.  These  questions 
will  be  used  to  guide  the  development  of  a  research  plan  for 
ARIARDA/Anacapa  computer-based  training  research. 
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TRAINING  EFFECTIVENESS  OF  AVIATION  PART-TASK  TRAINERS 
Dr.  Dudley  J.  Terrell,  Project  Director 


Background 

Many  aviation  tasks,  procedures,  and  skills  are  train- 
able  with  the  use  of  simulators  and  part-task  training 
devices  (Flexman  &  Stark,  1987) .  The  development  of  this 
training  technology  requires  knowledge  of  the  psychological 
principles  underlying  the  individual  learning  processes  and 
effective  instructional  strategies  for  various  kinds  of 
performance.  Several  operational  training  problems  at  the 
U.S.  Army  Aviation  Center  have  been  identified  by  previous 
research,  and  prototype  training  devices  have  been  developed 
to  address  these  problems  (Miles,  1987;  Miles  &  LaPointe, 
1986a,  1986b;  Ruffner,  1987)  .  While  these  prototype  training 
devices  may  remediate  existing  training  deficiencies,  they 
will  also  serve  as  vehicles  for  conducting  computer-based 
instructional  strategies  research  at  the  U.S.  Army  Research 
Institute  Aviation  Research  and  Development  Activity 
( ARIARDA) . 


The  development  of  prototype  devices  for  training 
research  is  an  evolutionary  process.  Preliminary  task 
analyr.es,  subject  matter  expert  advice,  and  existing  research 
are  used  to  design  the  first  systems.  At  several  stages  in 
the  design  process,  the  devices  must  be  subjected  to  training 
effectiveness  analyses.  The  results  of  these  analyses  are 
used  to  improve  the  design  of  the  training  system.  Research 
designed  to  evaluate  the  effectiveness  of  various  instruc¬ 
tional  strategies  provide  the  basis  for  making  recommen¬ 
dations  about  the  optimal  use  of  the  training  systems. 


Project  Objectives 

The  general  purpose  of  this  project  is  to  evaluate  the 
effectiveness  of  prototype  training  devices  and  instructional 
strategies  under  development  at  ARIARDA.  Specifically,  the 
project  is  designed  to  evaluate  the  training  effectiveness  of 
the  videodisc  versions  of  the  following  devices  in  a  manner 
that  elucidates  generally  effective  computer-based 
instructional  strateo-.es: 
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•  the  Advanced  Map  Interpretation  and  Terrain  Analysis 
Course  (MITAC) , 

•  the  Modernized  Cobra  Preflight  Inspection  Trainer,  and 

•  the  Basic  MITAC. 


Research  Approach 

Because  the  Advanced  MITAC,  the  Modernized  Cobra 
Preflight  Inspection  Trainer,  and  the  Basic  MITAC  programs 
are  at  different  stages  of  development,  a  general  research 
approach  is  presented.  The  status  of  the  research  is 
described  separately  for  each  program  in  the  next  section. 

To  evaluate  the  effectiveness  of  computer-based  instruc¬ 
tional  strategies  with  the  ARIARDA  part-task  training 
devices,  the  following  tasks  must  be  accomplished: 

•  design  training  effectiveness  experiments, 

•  conduct  experiments, 

•  feed  results  of  the  experiments  back  into  the  course¬ 
ware  design,  and 

•  continue  training  effectiveness  experimentation  until 
maximum  training  benefit  is  realized. 


Project-Status 


Advanced  J3II&C 

The  original  format  for  this  training  program  was  a  set 
of  16-mm  film  exercises  in  low-altitude  geographic  orien¬ 
tation  (Kelley,  1979) .  The  16-mm  films  have  been  converted 
to  videodisc,  and  the  exercises  have  been  upgraded  to  an 
interactive  computer-based  training  format  (Miles  &  LaPointe, 
1986b) .  The  current  research  and  development  effort  focuses 
on  (a)  the  general  effectiveness  of  the  interactive  videodisc 
method  for  training  geographic  orientation  skills,  and  (b) 
the  evaluation  of  computer-based  instructional  strategies. 

Work  completed.  An  experiment  was  conducted  to  evaluate 
the  training  effectiveness  of  the  Advanced  MITAC  and  to 
compare  the  effects  of  two  methods  of  computer-based  error 
remediation  on  inflight  navigation  performance.  Forty-one 
OH-58  student  pilots  were  given  an  inflight  contour-level 
navigation  pretest  at  the  beginning  of  the  Basic  Combat 
Skills  (BCS)  course.  The  students  were  assigned  to  one  of 
three  groups.  The  control  group,  with  13  subjects,  received 
only  the  standard  BCS  training.  The  two  experimental  groups, 
with  14  subjects  per  group,  received  supplemental  navigation 
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training  with  the  Advanced  MITAC  in  addition  to  the  standard 
BCS  training. 

The  two  experimental  groups  differed  only  in  the  manner 
in  which  Advanced  MITAC  errors  were  remediated.  For  one 
group,  errors  were  followed  by  a  computer  presentation  of  the 
correct  answer  and  a  brief  explanation  of  the  navigation 
strategy  that  would  have  produced  the  correct  answer  (i.e., 
errors  were  computer  remediated) .  For  the  other  group, 
errors  were  followed  by  a  requirement  to  rework  the  naviga¬ 
tion  problem  (i.e.,  errors  were  student  remediated) .  A 
control  group  of  13  subjects  received  only  the  standard  BCS 
training.  At  the  end  of  the  BCS  course,  all  subjects  were 
given  an  inflight  contour-level  navigation  posttest. 

A  signif icantly  larger  proportion  of  experimental 
subjects  than  control  subjects  performed  perfectly  on  an 
inflight  contour-level  posttest.  Of  the  experimental 
subjects  who  did  not  perform  perfectly,  those  in  the  student- 
remediation  group  tended  to  stray  slightly  farther  and  to 
spend  more  time  off  course  than  those  in  the  computer- 
remediation  group.  Advanced  MITAC  training  had  no  effect  on 
the  number  of  times  that  subjects  were  assisted  on  the  post¬ 
test  by  an  instructor  pilot,  nor  did  it  affect  the  distribu¬ 
tions  of  final  grades  for  Terrain  Flight  Navigation,  Map 
Interpretation,  or  the  BCS  checkride. 

The  findings  suggest  that  Advanced  MITAC  training  is 
effective  for  teaching  contour-level  navigation  skills  to 
helicopter  pilots.  The  findings  also  suggest  that  computer¬ 
generated  error  remediation  might  be  more  effective  than 
student-generated  error  remediation.  However,  these  findings 
require  confirmation  through  additional  research. 

A  draft  technical  report  (Terrell,  1988a)  was  submitted 
to  ARIARDA  for  review.  Additionally,  a  plan  for  follow-on 
research  (Terrell,  1988b)  was  submitted  to  ARIARDA. 

Work  projected.  Pending  approval  by  ARIARDA,  research 
will  be  conducted  to  evaluate  the  effects  of  film  speed, 
geographic  region,  and  lesson  sequence  on  Advanced  MITAC 
performance.  Further  research  will  be  designed  to  analyze 
the  effectiveness  of  the  Advanced  MITAC  as  a  skill  sustain¬ 
ment  trainer  and  to  investigate  whether  the  Advanced  MITAC 
exercises  can  be  used  to  diagnose  geographic  orientation 
skill  deficiencies.  The  Advanced  MITAC  also  will  be  used  as 
a  vehicle  for  the  planned  ARIARDA  research  on  computer-based 
instructional  strategies  (see  Survey  of  Research  in  Computer- 
Based  Instructional  Strategies,  pp.  99-102  of  this  report). 
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The  original  format  for  this  training  program  was  an 
interactive  videotape  controlled  by  a  microcomputer  (Miles  & 
LaPointe,  1986a)  .  Preliminary  research  demonstrated  the 
effectiveness  of  the  experimental  program  as  measured  by 
performance  on  a  preflight  inspection  multiple-choice  test 
(Intano,  1988).  However,  incidental  observations  during  this 
research  suggested  that  the  videotape  format  was  less  suit¬ 
able  than  videodisc  for  computer-based  training.  The  current 
research  effort  focuses  on  development  of  a  videodisc  version 
of  the  training  program  and  the  evaluation  of  computer-based 
instructional  strategies  with  the  videodisc  version  of  the 
trainer . 

Work  completed.  The  videotape  was  converted  to  a  video¬ 
disc  format,  and  initial  courseware  design  and  programming 
has  begun.  Software  has  been  developed  for  dual-screen 
viewing  of  the  videodisc  contents.  The  native  monitor  of  a 
host  computer  is  used  to  display  program  instructions  and 
computer-generated  graphics .  A  second  monitor  is  used  to 
display  video  from  the  videodisc  player.  The  program  senses 
the  graphics  hardware  available  on  the  host  computer  and 
adjusts  its  output  accordingly.  The  dual-screen  viewing 
software  does  not  include  any  interactive  courseware,  but 
does  permit  easy  and  organized  access  to  the  videodisc 
information . 

A  second  program  was  designed  and  developed  for  pre¬ 
senting  edited  information  from  the  videodisc.  This  program 
uses  a  single  switching  monitor  and  requires  a  special  video 
overlay  card  in  the  computer.  Computer-graphic  menus  are 
provided  using  the  organization  of  preflight  inspection 
checkpoints  specified  in  the  TM  55-1520-236-10  and  the  AH-1 
Prod  checklist.  The  student  interacts  with  the  computer  by 
operating  a  "mouse"  rather  than  typing  on  a  keyboard.  This 
version  of  the  training  program  currently  is  being  field 
tested  in  the  flightline  classrooms  of  the  AH-1  Aviator 
Qualification  Course  at  Fort  Rucker,  Alabama. 

Work  projected.  As  with  the  Advanced  MITAC,  the 
Preflight  Inspection  Trainer  will  be  used  as  a  vehicle  for 
conducting  the  planned  ARIARDA  research  on  computer-based 
instructional  strategies.  Experimental  courseware  will  be 
developed,  and  training  effectiveness  research  will  be 
conducted.  Future  research  and  development  will  utilize  the 
videodisc  format  in  an  analysis  of  computer-based  strategies 
for  training  crewchief  maintenance  of  the  AH-1.  Finally, 
research  will  be  conducted  to  analyze  the  effectiveness  of 
the  computer-based  videodisc  training  device  for  skill 


sustainment  training  in  operational  Army  aviation  units  and 
in  various  Army  Reserve  and  National  Guard  units.  The 
supportability  of  the  computer-based  videodisc  training 
device  in  operational  Army  aviation  units  and  in  various  Army 
Reserve  and  National  Guard  units  also  will  be  investigated. 


Basic  MITAC 

The  Basic  MITAC  is  currently  being  developed  by  Anacapa 
Sciences  under  a  different  project  title  (see  Development  of 
the  Basic  Map  Interpretation  and  Terrain  Analysis  Course,  pp. 
93-98  of  this  report) .  Upon  completion  of  that  project, 
training  effectiveness  evaluations  will  be  conducted  under 
the  present  project. 

Work  projected.  Experiments  will  be  designed  and 
initiated  to  evaluate  the  effectiveness  of  various  instruc¬ 
tional  strategies.  Different  computer-based  branching  routines 
for  remediating  knowledge  or  skill  deficiencies  and  different 
methods  for  presenting  drills  and  tutorials  will  be  compared. 
The  capabilities  of  the  videodisc  medium  will  be  evaluated, 
including  the  differential  effects  of  full-motion  and  still 
graphics,  digital  and  analog  imagery,  and  the  effects  of  audio 
narrative  on  knowledge  or  skill  acquisition.  In  addition,  the 
research  will  evaluate  the  effectiveness  of  the  Basic  MITAC  in 
training  general  map  interpretation  and  terrain  analysis  skills 
as  a  prerequisite  for  Advanced  MITAC  training. 
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TECHNICAL  ADVISORY  SERVICE:  LHX  SIMULATION  REQUIREMENTS 
Mr.  Theodore  B.  Aldrich,  Technical  Advisor 


Background 

As  part  of  its  aviation  modernization  program,  the  U.S. 
Army  plans  to  develop  and  acquire  a  new,  highly  combat- 
effective  and  affordable  helicopter  to  eliminate  projected 
deficiencies  in  the  light  helicopter  fleet.  The  proposed  new 
light  helicopter,  designated  the  LHX,  will  have  many  advanced 
technology  features  to  increase  its  effectiveness,  maintain¬ 
ability,  and  survivability. 

Anacapa  Sciences,  under  contract  to  the  U.S.  Army 
Research  Institute  Aviation  Research  and  Development  Activity 
(ARIARDA) ,  has  been  providing  technical  support  to  the  LHX 
program  since  July  1983.  Anacapa  and  ARIARDA  researchers 
developed  a  methodology  for  predicting  crew  workload  in 
emerging  aircraft.  They  applied  the  new  methodology  to 
predict  operator  workload  for  42  different  conceptual  designs 
of  the  LHX  featuring  various  automation  options  in  both  the 
one-crewmember  and  two-crewmember  configurations  (Aldrich, 
Szabo,  &  Craddock,  1986) . 

Concurrent  with  the  early  LHX  conceptual  design  activi¬ 
ties,  the  Army  contracted  for  a  multi-year,  Advanced  Rotor- 
craft  Technology  Integration  (ARTI)  development  program. 

Each  of  five  contractors  was  required  to  develop  an  advanced 
technology  mission  equipment  package  for  the  light  helicopter 
battlefield  mission  projected  for  the  year  2000.  In  addi¬ 
tion,  each  contractor  was  required  to  perform  a  detailed 
mission/task  analysis,  develop  or  adopt  a  workload  prediction 
methodology,  and  apply  it  to  the  operator  tasks  imposed  by 
the  proposed  mission  equipment  package  design. 

Because  of  the  expertise  gained  in  developing  and  exer¬ 
cising  the  LHX  workload  prediction  model,  Anacapa  researchers 
participated  in  the  evaluation  of  each  ARTI  contractor's 
developmental  efforts.  The  Anacapa  support  included  (a) 
visits  to  each  of  the  contractor  plants  to  help  evaluate  the 
full-mission  simulation  demonstrations,  and  (b)  participation 
in  all  ARTI  program  reviews  to  help  evaluate  the  human 
factors  aspects  of  the  proposed  designs. 

Within  the  past  year,  the  Army  has  modified  the  design 
goals  for  the  LHX  program.  The  new  proposed  LHX  design  is 
for  a  two-crewmember  aircraft  that  is  single-crewmember 
operable  from  one  crew  station  and  has  a  fly-home  capability 
in  the  other  crew  station.  There  are  also  changes  in  the 
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empty  design  weight  goal,  the  LHX  engine,  and  the  design-to- 
cost  goal.  The  first  activities  in  the  revised  LHX  system 
development  program  is  an  18-month  Demonstration/Validation 
(DEM/VAL)  phase.  For  the  DEM/VAL  phase,  the  Army  plans  to 
award  equally  funded  contracts  to  two  contractor  teams. 
Following  the  DEM/VAL  phase,  the  Army  anticipates  competitive 
selection  of  a  contractor  team  to  produce  the  LHX. 

As  part  of  the  DEM/VAL  program,  each  contractor  team  is 
required  to  conduct  engineering  simulations  of  their  (a) 
cockpit  automation  and  design  approaches,  (b)  mission  equip¬ 
ment  package/armament  system  integration,  (c)  manpower  and 
personnel  integration/training  initiatives,  (d)  flight/ 
handling  qualities,  and  (e)  mission  effectiveness.  Because 
of  Anacapa ' s  experience  with  the  earlier  LHX  and  ARTI 
programs,  ARIARDA  requested  that  the  Anacapa  technical 
advisor  support  the  revised  LHX  program  in  the  area  of  human 
factors  simulation  requirements. 


Support  Provided 

During  the  current  contract  year,  the  Anacapa  technical 
advisor  attended  three  meetings  concerning  the  LHX  simulation 
requirements.  The  first  meeting  was  conducted  at  the  U.S. 
Army  Aviation  Center  (USAAVNC) ,  Fort  Rucker,  Alabama,  on  25 
May  1988.  Prior  to  the  meeting,  the  Anacapa  technical 
advisor  reviewed  the  draft  "Light  Helicopter  System  (LHX) 
Demonstration/Validation  Request  for  Proposal"  and  the  draft 
"Light  Helicopter  System  (LHX)  Specification."  Topics 
discussed  at  the  first  meeting  included: 

•  whether  the  Army  should  impose  simulation  requirements 
on  the  LHX  contractors  similar  to  those  imposed  during 
the  ARTI  program, 

•  whether  the  Army  should  provide  the  contractors  with 
(a)  a  standard  mission  scenario,  (b)  Army  pilots  to 
serve  as  subjects,  and  (c)  a  standard  set  of  perfor¬ 
mance  measures  for  the  simulation  program, 

•  whether  a  standard  workload  prediction  methodology 
should  be  imposed  on  the  contractors,  and 

•  whether  a  simulation  evaluation  team  (SET)  of  Army 
experts  should  be  formed  to  evaluate  the  contractors' 
simulation  studies  and  results. 

The  discussion  continued  at  the  second  meeting,  which 
was  held  at  the  Crew  Station  Research  and  Development 
Facility  (CSRDF ) ,  NASA  Ames  Laboratory,  Moffett  Field, 
California,  on  26  and  27  July  1988.  During  this  meeting,  the 
LHX  Program  Office  indicated  that  a  standard  mission  scenario 
should  be  developed  and  provided  to  each  of  the  contractor 
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teams  to  make  the  simulation  results  comparable.  The 
attendees  decided  that  the  standard  scenario  also  should  be 
compatible  with  the  capabilities  of  the  CSRDF . 

An  approach  for  deriving  measures  of  effectiveness 
(MOEs)  from  the  submeasures  of  performance  collected  during 
simulations  was  discussed  at  this  second  meeting.  Although 
performance  will  probably  be  measured  at  the  task  level 
(e.g.,  maintaining  altitude),  the  decision  makers  are 
interested  in  more  global  MOEs  such  as  targets  destroyed, 
threats  identified,  and  the  use  of  battle  resources.  A 
computer-based  methodology  was  demonstrated  for  collecting 
subject  matter  expert  judgments  about  the  importance  of  each 
of  the  submeasures  to  the  respective  MOEs.  Attendees 
suggested  MOEs  and  submeasures  they  thought  appropriate .  The 
Anacapa  technical  advisor  suggested  the  following 
submeasures : 

•  (under  MOE,  Target  Interactions)  Engagement  Time  and 
Maintaining  Sight  of  Target  (Air-to-Air) , 

•  (under  MOE,  Piloting  and  Navigation)  Energy  Manage¬ 
ment,  and 

•  (under  MOE  Workload)  Task  Reallocation  Among  Crews  and 
Crew  Coordination  Errors. 

During  the  discussion,  it  was  learned  that  the  LHX 
Program  Manager  had  directed  that  a  SET  should  not  be  formed 
during  the  DEM/VAL  period  of  the  program.  However,  an 
evaluation  team  may  be  formed  in  conjunction  with  the  LHX 
Source  Selection  Evaluation  Board.  Therefore,  a  decision  was 
made  to  develop  a  plan  for  a  SET  composed  of  Army  evaluation 
pilots  and  government  evaluation  engineers  and  to  staff  the 
plan  within  the  LHX  program  office  and  the  Directorates  of 
Combat  Developments  at  the  USAAVNC  and  the  Training  and 
Doctrine  Command. 

The  third  meeting,  conducted  at  the  USAAVNC,  continued 
the  discussions  about  performance  measures  required  for  LHX 
developmental  simulation  programs.  Three  action  were  taken 
at  the  meeting  after  attack  helicopter  pilots  and  other 
subject  matter  experts  had  critiqued  the  list  of  MOEs  and 
submeasures.  First,  the  MOEs  were  divided  into  three  cate¬ 
gories:  Ground  Target  measures,  Air  Target  measures,  and 

Reconnaissance  measures.  Second,  the  Piloting  and  Navigating 
MOE  was  separated  into  two  distinct  MOEs,  one  for  Piloting 
and  another  for  Navigation.  Finally,  Degradation  in  Task 
Performance  was  added  to  the  list  of  submeasures  under  the 
Workload  MOE. 


ill 


Support,  Projected 

During  the  next  contract  year,  the  Anacapa  technical 
advisor  will  continue  to  support  LHX  developmental  simulation 
efforts  as  directed  by  ARIARDA.  At  the  close  of  the  current 
contract  year,  the  LHX  DEM/VAL  contracts  had  not  yet  been 
awarded.  No  additional  technical  support  requirements  are 
anticipated  by  the  technical  advisor  until  the  DEM/VAL 
contracts  are  awarded. 
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TECHNICAL  ADVISOR.  SERVICE:  SUPPORT  TO  THE 
SPECIAL  OPERATIONS  AIRCRAFT  PROGRAM  MANAGER'S  OFFICE 

Mr.  Carl  R.  Bierbaum,  Technical  Advisor 


Background 

The  U.S.  Army  provides  aviation  support  to  the  Special 
Operations  Forces,  a  Department  of  Defense  unit.  The  CH-47D 
and  UH-60A  aircraft  are  currently  being  utilized  for  aviation 
support.  However,  the  Army  is  developing  special  aircraft 
that  will  have  additional  capabilities.  The  Special  Opera¬ 
tions  Aircraft  (SOA)  Program  Manager's  (PM)  Office  at  the 
Army  Aviation  Systems  Command  has  been  tasked  to  develop  the 
MH-60K  and  MH-47E  aircraft.  These  aircraft  will  consist  of 
existing  CH-47D  and  UH-60A  airframes  with  a  new,  integrated 
cockpit.  A  standardized  integrated  cockpit  featuring  four 
multifunction  display  (MFD)  units  will  replace  the  present 
CH-47D  and  UH-60A  instrument  and  gauge  configurations. 

The  MH-60K  and  MH-47E  aircraft  are  being  designed  to 
provide  special  operations  pilots  with  aircraft  that  have 
increased  capabilities  and  reduced  crewmember  workload. 
However,  the  high  technology  modifications  being  proposed  for 
the  MH-60K  and  MH-47E  cockpits  may  increase  workload  by 
placing  additional  demands  on  the  mental  resources  of  the 
crewmembers.  Researchers  from  the  U.S.  Army  Research 
Institute  Aviation  Research  and  Development  Activity 
( ARIARDA)  and  Anacapa  Sciences  have  developed  a  methodology 
for  conducting  mission/task  analyses  and  predicting  workload 
for  emerging  systems  (Aldrich,  Szabo,  &  Craddock,  1986)  . 
Additionally,  Anacapa  researchers,  under  contract  to  ARIARDA, 
have  completed  a  research  product  entitled  "Task  Analysis  of 
the  UH-60  Mission  and  Decision  Rules  for  Developing  a  UH-60 
Workload  Prediction  Model"  (Bierbaum,  Szabo,  &  Aldrich, 

1987) .  Because  of  the  experience  gained  from  the  conduct  of 
these  projects,  Anacapa  has  been  tasked  with  providing 
technical  advisory  services  to  the  MH-60K  and  MH-47E 
development  programs. 


Support  Provided 

The  primary  technical  advisory  support  activities  during 
the  current  contract  year  consisted  of  participation  in: 

•  all  SOA  in-progress  reviews  (IPRs) ,  and 

•  all  SOA  Crew  Station  Working  Group  (CSWG)  meetings. 

The  Anacapa  technical  advisor  participated  in  the  following 
IPRs  : 
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•  System  Design  Review 

•  CSWG  meeting  #1 

•  CSWG  meeting  #2 

•  MH-60K  Preliminary  Design  Review 

•  CSWG  meeting  #3 

•  Hardware  Preliminary  Design  Review 

•  Software  Specification  Review 

•  CSWG  meeting  #4 

•  MH-47E  Critical  Design  Review 

•  Hardware  Critical  Design  Review/ 
Software  Preliminary  Design  Review 

•  MH-60K  Critical  Design  Review 


23-24 

Feb 

88 

28-31 

Mar 

88 

3-  6 

May 

88 

14-15 

Jun 

88 

22 

Jun 

88 

23-24 

Jun 

88 

12-14 

Jul 

88 

10-12 

Aug 

88 

16-18 

Aug 

88 

13-16 

Sep 

88 

27-29 

Sep 

88 

None  of  the  IPRs  resulted  in  a  delay  in  the  MH-47E/MH-60K 
development  programs. 


During  the  System  Design  Review  held  at  IBM  in  February 
1988,  a  CSWG  was  formed  to  assist  the  contractors  in 
resolving  SOA  design  issues.  The  CSWG  met  four  times  on  the 
following  dates:  28-31  March,  3-6  May,  22  June,  and  10-12 
August  1988.  During  the  first  meeting,  the  basic  system 
design  was  described  to  the  members.  The  briefing  included 
all  MFD  control  layer  and  Control  Display  Unit  functions  to 
ensure  that  the  CSWG  members  thoroughly  understood  the 
Integrated  Avionics  System  (IAS).  The  Anacapa  technical 
advisor  attended  all  four  meetings  and  contributed  the 
knowledge  and  expertise  gained  from  the  UH-60/CH-47 
Task/Workload  Analyses  project  as  the  CSWG  resolved  design 
issues.  The  major  issues  resolved  by  the  CSWG  are  summarized 
below . 


•  Although  the  IAS  has  redundancy  throughout  the  system, 
the  CSWG  recommended  that  the  following  instruments  be 
installed  as  backup  instruments: 

— attitude  indicator, 

— airspeed  indicator, 

--barometric  altimeter, 

— radar  altimeter, 

--turn  and  slip  indicator, 

--stabilized  compass,  and 
--accelerometer  (MH-60K  only) . 

•  When  the  MFD  displays  summary  pages  of  data  in  a 
matrix,  the  proposed  design  has  a  small  caret  at  the 
extreme  left  of  one  line  to  indicate  that  the  line  is 
active.  The  CSWG  recommended  that,  in  addition  to  the 
caret,  the  selected  line  be  underlined  to  reduce  the 
difficulty  in  identifying  the  intersections  of  columns 
and  lines. 
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•  In  the  SOA  design,  no  information  about  the  fuel 
available  in  the  auxiliary  tanks  is  provided  to  the 
IAS  for  computation.  As  a  result,  the  fuel  infor¬ 
mation  provided  to  the  pilot  by  the  IAS  is  inaccurate. 
The  CSWG  recommended  that  sensors  be  installed  in  all 
fuel  tanks  to  provide  total  fuel  available  information 
to  the  IAS. 

•  The  new  capabilities  of  the  MH-60K  and  MH-47E  required 
that  the  cyclic  and  collective  grip  configurations  be 
redesigned.  Additionally,  the  SOA  PM  required  that 
the  two  configurations  be  standardized  as  much  as  the 
aircraft  peculiarities  will  allow.  The  CSWG  members 
designed  the  MH-60K  and  MH-47E  cyclic  and  collective 
grip  configurations  and  provided  their  recommendations 
to  the  contractors. 

•  The  forward-looking  infrared  sensor  in  the  IAS  is 
controlled  by  a  common  tracking  handle  to  be  used  by 
the  pilot  or  copilot.  The  CSWG  members  analyzed 
optional  locations  of  the  tracking  handle  to  ensure 
easy  access  by  both  crewmembers  and  to  reduce  any 
obstruction  to  the  switch  panels.  Recommendations 
were  provided  for  both  the  MH-60K  and  MH-47E. 

•  The  SOA  will  have  four  different  modes  of  navigation 
available.  The  SOA  PM  requested  that  the  CSWG  decide 
which  design  option  should  be  provided  when  a  higher 
priority  mode  is  available  after  a  short  interruption. 
The  CSWG  recommended  that  the  IAS  should  advise  the 
pilot  of  the  higher  priority  mode  and  automatically 
upgrade  after  10  seconds,  with  the  pilot  being  able  to 
override  the  automatic  mode  change  during  the  10- 
second  delay. 


The  Anacapa  technical  advisor  will  attend  the  final 
prototype  IPR  (the  Software  Critical  Design  Review)  in 
December  1988.  The  technical  advisor  will  attend  future 
me  stings  of  the  CSWG  when  convened  by  the  SOA  PM.  Anacapa 
will  continue  to  provide  technical  advisory  support  to  the 
SOA  PM  when  tasked  by  ARIARDA. 
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TECHNICAL  ADVISORY  SERVICE:  SUPPORT  TO 
AH-64A  MODIFICATIONS 

Dr.  Sandra  M.  Szabo,  Mr.  Carl  R.  Bierbaum,  and 
Mr.  Theodore  B.  Aldrich,  Technical  Advisors 


Background 

In  1987,  the  Army  Aviation  Systems  Command  (AVSCOM) 
decided  to  modify  the  AH-64A  aircraft  to  provide  an  Airborne 
Target  Handover  System  (ATHS) .  The  ATHS  modification 
includes  the  redesign  and  integration  of  the  AH-64A  avionics. 
One  of  the  primary  operational  requirements  identified  for 
the  ATHS  Avionics  Integration  (ATHS/AI)  program  is  a  reduc¬ 
tion  in  the  amount  of  time  required  to  perform  certain  AH-64A 
mission  functions.  AVSCOM  requested  that  the  U.S.  Army 
Research  Institute  Aviation  Research  and  Development  Activity 
( ARIARDA)  conduct  research  to  assess  the  effect  that  the 
proposed  modifications  may  have  on  operator  workload.  In 
response  to  AVSCOM' s  request,  ARIARDA  tasked  Anacapa  Sciences 
to  conduct  the  required  research. 

Anacapa  had  previously  conducted  a  comprehensive  task 
analysis  of  the  AH-64A  mission.  The  analysis  provided 
workload  estimates  for  AH-64A  crewmembers  and  preliminary 
decision  rules  for  an  AH-64A  workload  prediction  model.  The 
model  serves  as  a  baseline  for  evaluating  modifications  to 
the  AH- 64 A .  The  results  of  that  research  are  reported  in 
Szabo  and  Bierbaum  (1986)  and  summarized  in  this  report  (see 
Development  and  Validation  of  the  AH-64A  Workload  Prediction 
Model,  pp .  23-29). 

Anacapa  scientists,  assisted  by  AH-64A  subject  matter 
experts,  supported  the  ATHS/AI  during  1987  by  conducting  an 
analysis  to: 

•  identify  the  crew  functions  in  the  AH-64  mission  that 
are  most  likely  to  be  affected  by  ATHS/AI, 

•  identify  crewmember  tasks  that  are  likely  to  be 
eliminated  by  ATHS/AI, 

•  identify  additional  and/or  replacement  tasks  that  are 
likely  to  be  required  by  ATHS/AI, 

•  determine  the  ATHS/AI  design  goal  timelines  for  the 
selected  functions,  and 

•  compare  the  ATHS/AI  design  goal  timelines  with  the 
baseline  timelines  for  the  selected  functions. 

The  AVSCOM  program  managers  adopted  the  Anacapa  design 
goal  timelines  and  included  them  in  the  Army's  ATHS/AI 
request  for  proposals.  The  results  of  the  Anacapa  ATHS 
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analysis  were  reported  in  Szabo,  Bierbaum,  and  Aldrich 
(1987)  . 


Support  Provided 

During  the  current  contract  year,  Anacapa  continued  to 
provide  technical  assistance  to  the  AH-64A  modification 
programs.  In  November  and  December  1987,  an  Anacapa  tech¬ 
nical  advisor  reviewed  the  timelines  and  workload  predictions 
and  provided  information  to  the  AVSCOM  Source  Selection 
Evaluation  Board  for  the  AH-64A  ATHS/AI.  The  information  was 
used  to  help  evaluate  the  contractors'  methods  for  evaluating 
workload  associated  with  their  proposed  ATHS/AI  designs.  The 
Anacapa  advisor  also  assisted  in  the  evaluation  of  function 
and  scenario  timelines  submitted  by  various  contractors. 

During  April  and  May  1988,  Anacapa  technical  advisors 
assisted  in  the  evaluation  of  74  engineering  change  proposals 
(ECPs)  for  the  Rockwell-Collins  ATHS .  The  ECPs  were  sub¬ 
mitted  by  the  U.S.  Army  Avionics  Research  and  Development 
Activity  (AVRADA)  at  Fort  Monmouth,  New  Jersey.  Anacapa 
advisors  reviewed  and  evaluated  the  ECPs  and  developed 
recommendations  for  each  of  the  AVRADA  proposals.  The 
evaluations  and  recommendations  were  presented  at  an  ATHS 
users  conference  at  Fort  Rucker  on  27  April.  The  Anacapa 
submission  was  subsequently  adopted  and  used  as  a  working 
paper  during  the  user's  conference. 

During  May  and  June  1988,  Anacapa  provided  technical 
advice  at  several  meetings  concerning  the  ATHS/AI.  The 
purposes  of  the  meetings  were: 

•  to  review  additional  ECPs  developed  by  Rockwell- 
Collins  International  Corporation  and  to  identify  the 
ECPs  that  should  be  recommended  for  the  ATHS; 

•  to  discuss  an  approach  for  using  the  AH-64A  ATHS  to 
validate  a  workload  prediction  methodology,  developed 
by  Analytics,  Inc.,  with  minimal  duplication  of  effort 
among  Analytics,  Rockwell-Collins,  and  Sikorsky 
Aircraft  Company;  and 

•  to  discuss  proposed  modifications  during  each  phase  of 
an  AH- 64 A  Multi-Stage  Improvement  Program  proposed  by 
the  McDonnell-Douglas  Helicopter  Company. 


Support  Projected 

During  the  next  contract  year,  Anacapa  will  continue  to 
support  modification  programs  for  the  AH-64A  as  directed  by 
ARIARDA. 
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TECHNICAL  ADVISORY  SERVICE:  SUPPORT  TO 
MULTITRACK  TESTING 

Dr.  D.  Michael  McAnulty,  Mr.  Gary  Coker, 

Ms.  Cassandra  Hocutt,  Ms.  Stephanie  Noland,  and 
Mr.  Kenneth  Persin,  Technical  Advisors 


Background 

The  U.S.  Army  Research  Institute  Aviation  Research  and 
Development  Activity  (ARIARDA)  at  Fort  Rucker,  Alabama,  is 
conducting  research  designed  to  develop  a  classification 
algorithm  for  use  in  the  Initial  Entry  Rotary  Wing  (IERW) 
Course.  The  purpose  of  the  algorithm  is  to  assign  flight 
students  into  one  of  several  mission  tracks  (AH-1,  CH-47, 
OH-58,  UH-1 ,  or  UH-60)  at  an  early  stage  of  IERW  training. 
During  the  current  contract  year,  ARIARDA  tasked  Anacapa 
Sciences  to  provide  various  types  of  research  support  for  the 
Multitrack  algorithm  development. 


Research  Approach 

The  ARIARDA  research  approach  involves  three  phases. 
First,  ARIARDA  compiled  or  adapted  existing  paper-and-pencil 
and  computer  administered  tests  that  may  be  related  to 
aviator  performance  into  an  experimental  battery.  Second, 
the  tests  were  administered  to  high-time  aviators  in  each 
track  and  to  IERW  students  to  develop  an  experimental 
prediction  algorithm  using  a  subset  of  the  tests.  Third,  the 
experimental  algorithm  is  being  validated  by  administering 
the  tests  to  entering  IERW  students,  classifying  them  into 
mission  tracks,  and  then  evaluating  their  performance  in 
training . 


Support  Prpy.i.de£i 

During  the  current  contract  year,  Anacapa  personnel 
provided  the  following  types  of  support  to  the  Multitrack 
Testing  project: 

•  electronic  technician  support  in  assembling  and 
maintaining  the  test  apparatus, 

•  software  support  in  developing  the  computerized  test 
programs , 

•  test  administration  support  during  the  second  phase  of 
the  research, 

•  data  entry  and  verification  support  following  test 
administration,  and 
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statistical  analysis  support  during  the  development  of 
the  validation  algorithm. 


Anacapa  personnel  will  continue  to  provide  support  to 
the  Multitrack  Testing  project  as  required.  Because  the 
research  is  now  in  the  third  phase,  most  of  the  support  will 
be  in  equipment  maintenance,  data  processing,  and  statistical 
analysis . 
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TECHNICAL  ADVISORY  SERVICE:  SUPPORT  TO 
HANDEDNESS  AND  IERW  PERFORMANCE 

Dr.  D.  Michael  McAnulty,  Technical  Advisor 


Background 

In  August  1987,  Anacapa  Sciences  was  tasked  to  provide 
technical  support  to  a  research  project  investigating  the 
relationship  between  handedness  and  the  performance  of 
helicopter  flight  students  in  the  Initial  Entry  Rotary  Wing 
(IERW)  Course  conducted  at  Fort  Rucker,  Alabama.  The 
research  project  was  being  conducted  under  the  auspices  of 
the  U.S.  Air  Force  School  of  Aerospace  Medicine  (SAM),  Brooks 
Air  Force  Base,  Texas.  The  project  was  expected  to  dovetail 
with  the  new  Flight  Aptitude  Selection  Test  (NFAST)  project. 


Research  Approach 

The  research  approach  was  a  straightforward  predictive 
validity  study.  Students  were  administered  a  standardized 
handedness  inventory  during  the  early  phases  of  IERW.  Subse¬ 
quently,  performance  data  were  collected  and  correlated  with 
the  results  of  the  handedness  scores.  The  Anacapa  technical 
advisor  was  asked  to  provide  (a)  information  about  IERW 
training,  (b)  advice  on  experimental  design  and  statistical 
analysis,  (c)  biographical  information  on  the  tested  students, 
and  (d)  IERW  performance  data.  The  SAM  project  director  made 
separate  arrangements  to  administer  the  handedness  inventory 
and  to  conduct  the  analyses . 


Support  Provided 

The  technical  advisor  has  completed  Anacapa' s  portion  of 
the  project  unless  further  support  is  requested.  Usable 
handedness  inventories  were  completed  by  190  students.  The 
technical  advisor  provided  biographical  data  for  155  students 
who  also  participated  in  the  NFAST  research.  The  technical 
advisor  obtained  IERW  performance  data  (primary,  transition, 
basic,  instrument,  and  advanced  instrument  phase  flight 
grades;  and  the  overall  average  grades  at  the  end  of  the 
advanced  instrument  phase)  for  185  of  the  tested  students. 

No  data  could  be  obtained  on  the  other  5  students.  Through¬ 
out  the  project,  the  technical  advisor  provided  assistance  on 
IERW  training,  experimental  design,  and  statistical  analyses 
as  required. 
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The  final  activity  was  to  critique  a  draft  project 
report.  The  results  indicate  a  significant  relationship 
between  handedness  and  phase  flight  grades  in  primary  and 
basic  instrument  training.  In  general,  right-handers  per¬ 
formed  better  in  primary  and  left-handers  performed  better  in 
basic  instrument  training.  However,  the  total  variance 
accounted  for  by  these  relationships  is  very  small. 
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TECHNICAL  ADVISORY  SERVICE:  SOFTWARE  DEVELOPMENT  FOR 
FLIGHTLINE  RESEARCH  SYSTEMS 

Ms.  Stephanie  M.  Noland,  Technical  Advisor 

Background 

Scheduling  of  aviation  training  resources  at  the  U.S. 
Army  Aviation  Center  (USAAVNC) ,  Fort  Rucker,  Alabama,  is  a 
time-consuming  and  potentially  costly  activity.  For  example, 
the  l-14th  Battalion  currently  coordinates  and  schedules 
requests  for  aircraft,  stagefields,  refueling,  bus  and  air 
transportation,  ammunition,  and  remote  landing  site  lighting. 
The  scheduling  of  these  resources  is  accomplished  manually;  a 
mistaken  order  of  only  one  extra  aircraft  can  cost  hundreds 
of  dollars. 

Currently,  the  resources  required  for  all  classes  are 
scheduled  in  five-week  written  projections,  one-week  updated 
written  projections,  and  two  daily  squaring  sessions  con¬ 
ducted  by  telephone.  The  Army  currently  uses  Form  325  for 
this  scheduling. 


Project  Objective 

The  objective  of  this  project  is  to  develop  a  computer 
hardware  and  software  system  that  will  automate  the 
scheduling  of  aviation  training  resources.  An  efficient, 
automated  system  will  increase  speed  and  accuracy  and  reduce 
the  costs  of  scheduling  errors.  The  U.S.  Army  Research 
Institute  Aviation  Research  and  Development  Activity 
(ARIARDA)  contracted  with  another  contractor  for  development 
of  the  automated  system.  ARIARDA  tasked  Anacapa  Sciences  to 
provide  software  development  support  to  the  prime  contractor 
during  the  final  weeks  of  the  project. 


Project  Approach 

A  MicroVax  II  (VAX)  microcomputer  was  chosen  as  the 
central  processor  for  the  scheduling  system.  Six  training 
companies  and  the  battalion  staff  were  provided  a  Zenith  248 
personal  computer  for  their  use  in  logging  onto  the  VAX  and 
for  accessing  the  resident  programs  and  data.  The  system 
design  provides  customized  programs  enabling  each  company  to 
build,  edit,  update,  and  print  data  about  their  classes.  At 
the  battalion  level,  computer  programs  gather  and  organize 
the  information  from  the  six  on-line  company  class  files  into 
battalion  reports.  The  battalion  reports  include  aircraft, 
ammunition,  stagefield,  refueling,  bus  transportation,  and 
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air  transportation  requests  for  each  company.  A  battalion 
program  gathers  the  twice  daily  company  change  requests  into 
a  transaction  log  to  square  with  the  brigade  schedules. 

The  l-14th  Battalion  at  the  USAAVNC  was  chosen  as  the 
prototype  training  organization  for  this  project  because  it 
has  the  most  complex  resource  requirements.  The  l-14th 
Battalion  has  six  training  companies  designated  with  letters 
A  -  F.  Each  company  manages  between  two  and  five  different 
courses.  As  many  as  seven  classes  may  be  in  residence  in  a 
company  at  any  given  time. 


Support  .Bjovided 

The  work  to  automate  the  scheduling  process  began  in 
February  1988.  Anacapa  was  tasked  to  assist  with  the  work  in 
July  of  1988.  Between  July  and  October,  the  Anacapa  techni¬ 
cal  advisor  provided  the  following  support. 


Software  Problem  Diagnosis  and  ...Correct ion 

The  original  automated  daily  squaring  program  required 
two  hours  to  run,  whereas  the  daily  squaring  could  be  per¬ 
formed  manually  in  30  minutes.  The  Anacapa  technical  advisor 
suggested  an  automated  log  for  recording  transactions  when 
the  companies  make  their  daily  changes.  The  transactions  are 
sorted  by  resource  affected  and  automatically  recorded  in  a 
battalion  log.  The  battalion  staff  can  use  this  log  to 
square  the  requests  within  the  required  time.  The  developers 
have  implemented  this  plan  for  one  training  company  to  date. 


Saf.twa.re  . Development 

The  Anacapa  technical  advisor  wrote  the  battalion  level 
software  program  for  consolidating  the  stagefield  and 
refueling  requests  from  all  classes  in  all  companies  and  for 
entering  the  requests  onto  the  appropriate  form  for  each 
field  requested.  The  forms  were  designed  to  match  the 
existing  battalion  Form  169.  The  program  has  been  accepted 
by  the  l-14th  Battalion. 

The  Anacapa  technical  advisor  subsequently  enhanced  the 
battalion  stagefield  and  refuel  request  software  program  to 
allow  for  (a)  two  stagefields  or  two  refueling  points  to  be 
requested  by  a  class  in  a  single  flight  period,  (b)  minimum 
stagefield  requests,  and  (c)  Period  5  or  Period  6  requests. 
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The  Anacapa  technical  advisor  obtained  the  course  flows 
for  each  company  (B  -  F)  and  developed  the  course  templates. 
A  VAX  account  was  established  for  each  company  (B  -  F)  with 
customized  programs  to  operate  for  their  respective  courses. 
The  programs  were  tested  using  sample  classes  and  demon¬ 
strated  to  Army  contacts  from  each  company. 


Battalion  Program  Modification 

At  the  end  of  the  current  contract  year,  the  Anacapa 
technical  advisor  was  assisting  the  developers  in  modifying 
each  battalion  program  to  provide  the  A  company  request 
processing  capability  to  Companies  B  -  F. 


Support.  Projected 

Work  to  be  completed  during  the  next  contract  year  will 
include : 

•  testing  the  full  company  and  battalion  software 
capabilities, 

•  bringing  all  the  companies  on-line, 

•  writing  user  and  system  documentation, 

•  training  the  users  from  the  battalion  and  each 
company, 

•  developing  a  method  of  software  change  control, 

•  enhancing  current  software, 

•  providing  system  operator  support,  and 

•  providing  ongoing  user  support  and  troubleshooting. 
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